Serving long-context LLMs is costly because attention computation grows linearly with context length. Dynamic sparse attention algorithms (DSAs) mitigate this by attending only to the key-value (KV) cache of critical tokens. However, with DSAs, the main performance bottleneck shifts from HBM bandwidth to HBM capacity: KV caches for unselected tokens must remain in HBM for low-latency decoding, constraining parallel batch size and stalling further throughput gains. Offloading these underutilized KV caches to DRAM could free HBM capacity, allowing larger parallel batch sizes. Yet, achieving such hierarchical HBM-DRAM storage raises new challenges, including fragmented KV cache access, HBM cache contention, and high HBM demands of hybrid batching, that remain unresolved in prior work.

Similarity-based vector search, which retrieves the most similar vectors to a given query vector from a large vector dataset, underlies many applications such as search, recommendation, and Large Language Models (LLMs). Some systems run vector search on GPUs to enjoy GPU's high parallelism, but we observe that they are limited in query throughput and latency. In particular, their query-centric GPU kernel conducts computation independently for each query, failing to reuse data loaded to the GPU shared memory across queries and leading to a low GPU compute utilization. While their batch-based task reordering rearranges computation for queries in a batch to reduce CPU-GPU data transfer, but latency is prolonged since each query needs to wait for its slowest task. To tackle these problems, we propose Hitcher. Specifically, to reuse data across queries and improve GPU utilization, Hitcher implements a cluster-centric GPU kernel to batch computation on the same data for multiple queries. To reduce query latency, Hitcher adopts the hitch-ride ordering, which preserves the arrival order for query processing while batching computation across queries to improve efficiency. Hitcher can also offload computation tasks to the CPU to reduce CPU-GPU data transfer and utilize multiple GPUs. Experimental results show that Hitcher achieves up to 22× lower P99 query latency and 9× higher query throughput when compared with the state-of-the-art GPU-based vector query processing systems.

This paper presents a novel hallucination detection method based on the internal states and output probabilities of large language models (LLMs) to address the common issue of hallucinations in model-generated content. We designed a new detection framework that extracts internal features such as hidden layer states and word probabilities, and analyzes the output probability distribution to more accurately identify potential hallucinations. Experimental results show that our method outperforms traditional hallucination detection approaches across multiple evaluation metrics and performs well on our EpicQA dataset. This method offers a new perspective for hallucination detection and makes a significant contribution to improving the reliability of language models.

Embedding-based recommendation models (ERMs) require large memory to host huge embedding tables and involve massive data traffic to read the embeddings. As a new interconnect, CXL suits ERMs since it can scale up single-machine memory with performant remote memory devices. However, directly running DRAM-based ERM serving systems on CXL yields poor performance because the bandwidth of CXL is notably lower than DRAM and can be easily saturated, making CXL memory the bottleneck. The non-uniform memory access (NUMA) architecture in modern CXL servers further decreased the system performance. In this paper, we design Carina for ERM serving on heterogeneous memory with CXL by considering such bandwidth asymmetry. In particular, Carina balances the memory access from different memory devices by storing hot embeddings with high access frequencies on DRAM and specifying the placement of embedding tables on the NUMA nodes. Moreover, Carina adopts bandwidth-aware task execution, which decomposes each batch of ERM requests into fine-grained tasks and schedules the tasks to control the real-time utilization of CXL bandwidth to avoid instantaneous saturation. We evaluate Carina under real CXL devices and find that it outperforms a CXL-oblivious baseline by an average of 5.38x and 4.04x in system throughput and request latency, respectively.

In view of the phenomenon that DDoS attacks of smart contract in permissioned blockchain cannot be well detected at present, we proposes a traffic anomaly detection method based on the two-layer GRU model. Firstly, we analyzed the traffic characteristics of DDoS attacks against permissioned blockchains, and the traffic characteristics are evaluated by random forest to obtain the importance ranking of the features, and the most important features are used to construct the dataset. Secondly, the traditional GRU model and encoder-decoder model are combined to reduce the dimension of the data through encoder, and then the data is restored through the decoding layer. Finally, experiments show that the two-layer GRU model has a high identification accuracy and accuracy rate in the detection of DDoS attacks.

In modern software development, there is a significant amount of reused code in popular software development kits (SDKs). The extensive reuse of third-party SDKs by developers during the development of large, heterogeneous Internet of Things (IoT) devices exacerbates the potential for potential security vulnerabilities and their subsequent mass propagation. One possible way to address these vulnerabilities is to employ cross-platform firmware vulnerability detection techniques, especially those rooted in the concept of similarity. One such technique, graph similarity detection, has become a focal point and has attracted a great deal of attention in the cybersecurity community. Graph Convolutional Networks (GCNs) are deep learning models specifically designed to process graph data. While traditional convolutional neural networks perform convolutional operations on regular grid data (e.g., images), GCN performs convolutional operations on graph data. It has shown excellent performance in many tasks on graph data such as node classification, graph classification and link prediction. In this paper, we propose a PFGCN framework that performs cross-layer graph convolutional network computations by providing an abstract representation of the procedure. Experimental results demonstrate the effectiveness of our scheme for cross-architecture firmware similarity detection.

Graph embedding learning computes an embedding vector for each node in a graph and finds many applications in areas such as social networks, e-commerce, and medicine. We observe that existing graph embedding systems (e.g., PBG, DGL-KE, and Marius) have long CPU time and high CPU-GPU communication overhead, especially when using multiple GPUs. Moreover, it is cumbersome to implement negative sampling algorithms on them, which have many variants and are crucial for model quality. We propose a new system called GE 2 , which achieves both generality and efficiency for graph embedding learning. In particular, we propose a general execution model that encompasses various negative sampling algorithms. Based on the execution model, we design a user-friendly API that allows users to easily express negative sampling algorithms. To support efficient training, we offload operations from CPU to GPU to enjoy high parallelism and reduce CPU time. We also design COVER, which, to our knowledge, is the first algorithm to manage data swap between CPU and multiple GPUs for small communication costs. Extensive experimental results show that, comparing with the state-of-the-art graph embedding systems, GE 2 trains consistently faster across different models and datasets, where the speedup is usually over 2x and can be up to 7.5x.

Knowledge graph reasoning (KGR) answers logical queries over a knowledge graph (KG), and embedding-based KGR (EKGR) becomes popular recently, which embeds both queries and KG entities such that the vector embeddings of a query and its answer entities are similar. Compared with traditional KGR methods based on subgraph matching, EKGR produces fewer intermediate results and is more robust to missing and noisy information in the KG. However, existing systems are inefficient for serving online EKGR queries because they can only batch queries of the same type for execution (i.e., query-level batching ) and hence have limited batching opportunities due to the heterogeneity of queries. To serve EKGR queries efficiently, we propose the Atom system with operator-level batching, which decomposes queries into operators and batches operators of the same type from different queries for execution. The insight is that the types of operators are far fewer than the types of queries, and thus different queries typically share common operators, yielding more batching opportunities. To schedule the operators, Atom adopts a hybrid policy, which improves system throughput and avoids starving rare operators. For efficiency, Atom incorporates system optimizations including two-level pipeline, opportunistic submission, pre-allocated memory buffer, and tailored GPU kernels. Experiment results show that compared with existing systems, Atom can improve query throughput by over 20x and reduce query latency by over 5x. Micro experiments suggest that the designs and optimizations are effective in improving system performance.

Blockchain is widely applied in many applications. However, it faces many security risks. Unlike external attacks, the evildoing of internal members need more attentions in permissioned blockchain. We introduce the malicious resource consumption attack which seriously degrade the performance of permissioned blockchain system. To detect malicious resource consumption attacks, we propose MRC-Dec which applies a LSTM-AE model to analyze traffic between blockchain nodes, to determine whether the attacks happen. First, we construct the traffic dataset for attack detection by implementing two types of malicious resource consumption, namely useless endorsement and storage pollution on Fabric. Next in order not to be affected by the blockchain scale which means the number of nodes in blockchain, the traffic features are extracted between the node sets instead of between nodes. After feature extraction, a sequence of traffic features in temporal order is obtained, which is input to the LSTM-AE model. The LSTM-AE model learns the characteristic of normal traffic, and a threshold is set to determine normal and attack. The evaluation results show that MRC-Dec can find the malicious resource consumption attacks with high performance and can well adopt to the blockchain scale.

Jointly utilizing multiple GPUs to train graph neural networks (GNNs) is crucial for handling large graphs and achieving high efficiency. However, we find that existing systems suffer from high communication costs and low GPU utilization due to improper data layout and training procedures. Thus, we propose a system dubbed Distributed Sampling and Pipelining (DSP) for multi-GPU GNN training. DSP adopts a tailored data layout to utilize the fast NVLink connections among the GPUs, which stores the graph topology and popular node features in GPU memory. For efficient graph sampling with multiple GPUs, we introduce a collective sampling primitive (CSP), which pushes the sampling tasks to data to reduce communication. We also design a producer-consumer-based pipeline, which allows tasks from different mini-batches to run congruently to improve GPU utilization. We compare DSP with state-of-the-art GNN training frameworks, and the results show that DSP consistently outperforms the baselines under different datasets, GNN models and GPU counts. The speedup of DSP can be up to 26x and is over 2x in most cases.

Subgraph matching is one of the most important problems in graph analytics. Many algorithms and systems have been proposed for subgraph matching. Most of these works follow Ullmann's backtracking approach as it is memory-efficient in handling an explosive number of intermediate matching results. However, they have largely overlooked an intrinsic problem of backtracking, namely repeated computation, which contributes to a large portion of the heavy computation in subgraph matching. This paper proposes a subgraph matching system, Circinus, which enables effective computation sharing by a new compression-based backtracking method. Our extensive experiments show that Circinus significantly reduces repeated computation, which transfers to up to several orders of magnitude performance improvement.

Aiming at the urgent need for a cross-domain authentication scheme with low computational cost, low latency, and high security in the edge computing environment, this paper proposes a blockchain-based cross-domain authentication protocol. In this protocol, the IoT devices in the edge computing environment themselves control the generation, encrypted storage, registration, use or cancellation of their identities, and the authenticity and credibility of these identities can be guaranteed through a trusted identity checking mechanism. The public information of these identities is shared among edge nodes of different domains using the consensus mechanism of blockchain. IoT devices can perform cross-domain authentication at the edge. The authentication process not only makes use of the advantages of low latency of edge computing, but also minimizes the disclosure of detailed identity information. Security analysis shows that the protocol can resist common attacks such as DDoS Attacks and Impersonation Attacks. Compared with several previous cross-domain authentication schemes, our protocol is more secure. Besides, this protocol has the advantage of low computational cost, suitable for low-performance IoT devices.

As a distributed ledger technology, the addition of new members in permissioned blockchain is usually composed of several steps among distributed nodes. The addition can not be considered successful until all of the steps are completed. In other words, these steps are an atomic operation. However, there is no solution for the atomic operation in existing permissioned blockchain, leading to an inconsistent state when the addition of new members is partially completed. To implement the atomic member addition in permissioned blockchain, we propose a method targeting at the atomic addition of new members based on distributed and autonomous rollback. After member addition starts, distributed nodes of existing members detect the new node and decide whether to rollback or not, instead of getting commands from the coordinator. After deciding to rollback, a new configuration block is added to achieve rollback of the uncompleted member addition. In order for the new configuration block to pass the policy validation of orderers, we set a rollback mode for orderers. The evaluation results show that our method can actually implement atomic member addition and has little impact on performance.

Graph neural networks (GNNs) have shown to significantly improve graph analytics. Existing systems for GNN training are primarily designed for homogeneous graphs. In industry, however, most graphs are actually heterogeneous in nature (i.e., having multiple types of nodes and edges). Existing systems train a heterogeneous GNN (HetGNN) as a composition of homogeneous GNN (HomoGNN) and thus suffer from critical limitations such as lack of memory optimization and limited operator parallelism. To address these limitations, we propose HGL - a heterogeneity-aware system for GNN training. At the core of HGL is an intermediate representation, called HIR, which provides a holistic representation for GNNs and enables cross-relation optimization in HetGNN training. We devise tailored optimizations on HIR, including graph stitching, operator fusion and operator bundling. Compared with DGL and PyG, HGL achieves a speedup from 7 to 22 times for training HetGNNs.

Subgraph matching is a fundamental building block in graph analytics. Due to its high time complexity, GPU-based solutions have been proposed for sub graph matching. Most existing GPU-based works can only cope with relatively small graphs that fit in GPU memory. To support efficient subgraph matching on large graphs, we propose a view-based method to hide communication overhead and improve GPU utilization. We develop VSGM, a sub graph matching framework that supports efficient pipelined execution and multi-GPU architecture. Ex-tensive experimental evaluation shows that VSGM significantly outperforms the state-of-the-art solutions.

Memory disaggregation is a promising technique in datacenters with the benefit of improving resource utilization, failure isolation, and elasticity. Hashing indexes have been widely used to provide fast lookup services in distributed memory systems. However, traditional hashing indexes become inefficient for disaggregated memory, since the computing power in the memory pool is too weak to execute complex index requests. To provide efficient indexing services in disaggregated memory scenarios, this article proposes RACE hashing, a one-sided RDMA-Conscious Extendible hashing index with lock-free remote concurrency control and efficient remote resizing. RACE hashing enables all index operations to be efficiently executed by using only one-sided RDMA verbs without involving any compute resource in the memory pool. To support remote concurrent access with high performance, RACE hashing leverages a lock-free remote concurrency control scheme to enable different clients to concurrently operate the same hashing index in the memory pool in a lock-free manner. To resize the hash table with low overheads, RACE hashing leverages an extendible remote resizing scheme to reduce extra RDMA accesses caused by extendible resizing and allow concurrent request execution during resizing. Extensive experimental results demonstrate that RACE hashing outperforms state-of-the-art distributed in-memory hashing indexes by 1.4–13.7× in YCSB hybrid workloads.

As an indispensable part of Hyperledger Fabric application system, smart contracts are mostly developed in general-purpose programming languages such as Golang. However, these smart contracts often have potential risks that cause serious problems such as failed transactions or sensitive information leakage on the ledger. Although there are already some detection tools for potential risks, e.g., Chaincode Scanner and Chaincode Analyzer, the accuracy and coverage of them are limited. In response to the above problems, this paper summarizes 16 potential risks in smart contracts, including three type risks: Non-determinism Risk, Logical Security Risk, and Private Data Security Risk. In order to detect them, we propose a new static analysis method based on Abstract Syntax Tree Analysis, Package Dependency Analysis, and Functional Dependency Analysis. Based on the new method, a detection system is designed that can detect 16 potential risks in smart contracts developed using Golang language more accurately and provide development suggestions to eliminate these risks.

Third-party IME (Input Method Editor) apps are often the preference means of interaction for Android users’ input. In this paper, we first discuss the insecurity of IME apps, including the Potentially Harmful Apps (PHAs) and malicious IME apps, which may leak users’ sensitive keystrokes. The current defense system, such as I-BOX, is vulnerable to the prefix substitution attack and the colluding attack due to the post-IME nature. We provide a deeper understanding that all the designs with the post-IME nature are subject to the prefix-substitution and colluding attacks. To remedy the above post-IME system’s flaws, we propose a new idea, pre-IME, which guarantees that “Is this touch event a sensitive keystroke?” analysis will always access user touch events prior to the execution of any IME app code. We design an innovative TrustZone-based framework named IM-Visor which has the pre-IME nature. Specifically, IM-Visor creates the isolation environment named STIE as soon as a user intends to type on a soft keyboard, then the STIE intercepts,Android event sub translates and analyzes the user’s touch input. If the input is sensitive, the translation of keystrokes will be delivered to user apps through a trusted path. Otherwise, IM-Visor replays non-sensitive keystroke touch events for IME apps or replays non-keystroke touch events for other apps. A prototype of IM-Visor has been implemented and tested with several most popular IMEs. The experimental results show that IM-Visor has small runtime overheads.

Third-party IME (Input Method Editor) apps are often the preference means of interaction for Android users' input. In this paper, we first discuss the insecurity of IME apps, including the Potentially Harmful Apps (PHA) and malicious IME apps, which may leak users' sensitive keystrokes. The current defense system, such as I-BOX, is vulnerable to the prefix-substitution attack and the colluding attack due to the post-IME nature. We provide a deeper understanding that all the designs with the post-IME nature are subject to the prefix-substitution and colluding attacks. To remedy the above post-IME system's flaws, we propose a new idea, pre-IME, which guarantees that "Is this touch event a sensitive keystroke?" analysis will always access user touch events prior to the execution of any IME app code. We designed an innovative TrustZone-based framework named IM-Visor which has the pre-IME nature. Specifically, IM-Visor creates the isolation environment named STIE as soon as a user intends to type on a soft keyboard, then the STIE intercepts, translates and analyzes the user's touch input. If the input is sensitive, the translation of keystrokes will be delivered to user apps through a trusted path. Otherwise, IM-Visor replays non-sensitive keystroke touch events for IME apps or replays non-keystroke touch events for other apps. A prototype of IM-Visor has been implemented and tested with several most popular IMEs. The experimental results show that IM-Visor has small runtime overheads.