Although applications involving long-context inputs are crucial for the effective utilization of large language models (LLMs), they also result in increased computational costs and reduced performance. To address this challenge, we propose an efficient, training-free prompt compression method that retains key information within compressed prompts. We identify specific attention heads in transformer-based LLMs, which we designate as evaluator heads, that are capable of selecting tokens in long inputs that are most significant for inference. Building on this discovery, we develop EHPC, an Evaluator Head-based Prompt Compression method, which enables LLMs to rapidly "skim through'' input prompts by leveraging only the first few layers with evaluator heads during the pre-filling stage, subsequently passing only the important tokens to the model for inference. EHPC achieves state-of-the-art results across two mainstream benchmarks: prompt compression and long-context inference acceleration. Consequently, it effectively improves performance with the reduced costs associated with commercial API calls compared to prompt compressing methods. We further demonstrate that EHPC attains competitive results compared to key-value cache-based acceleration methods, thereby highlighting its potential to enhance the efficiency of LLMs for long-context tasks.

The ultra-dense low-earth orbit (LEO) satellite networks (ULSNs) have become an important component of next-generation (6G) wireless networks, offering large-scale coverage and high-capacity service. Unlike traditional terrestrial network backbones deployed in closed, protected environments, satellite networks are exposed to highly dynamic environments, where space environment interference can severely affect channel conditions. This paper addresses the impact of sun outages, one of the most significant spatial interference factors, on satellite communication and proposes an adaptive coding and modulation scheme that dynamically adjusts the modulation and coding schemes (MCSs) based on real-time channel conditions to enhance the performance and communication quality of ULSNs. First, we model the channel environment of satellite-terrestrial microwave and inter-satellite laser links under sun outage interferences in ULSNs, which involves sun outage occurrence prediction and their interference quantification. Subsequently, to implement the ACM scheme, we use the seasonal autoregressive integrated moving average (SARIMA) algorithm combined with the bidirectional long short-term memory (BiLSTM) algorithm for time-series prediction of the channel state. Based on this, we apply the knowledge distillation-assisted proximal policy optimization (KD-PPO) algorithm to select the appropriate MCS. Simulation results show that by calculating the sun outage duration and the resulting interference, as well as predicting the channel state, the proposed KD-PPO algorithm can minimize the bit error rate (BER) and maximize the spectrum utilization (SU) in ULSNs.

We consider a model of broadcast channel where a pair of base-user operates in the full-duplex mode. A partial decode-forward strategy together with Marton's coding are adopted to obtain an inner bound. An outer bound is also presented. Numerical evaluations are performed on a particular set of discrete memoryless channels to compare the sum-rates of these two bounds and the one of time division duplex.

Graph partitioning is crucial in distributed graph-parallel computing systems, and it is challenging for graph partitioning to optimize the communication cost and load balancing together. Existing state-of-the-art works, such as Powerlyra and TopoX, optimize the load balancing by randomly distributing the edges of high-degree vertices, which inevitably brings a high communication cost that is unbounded. This paper proposes a graph partition model that can minimize communication cost while maximizing load balancing. More specifically, we model the graph partition as the combinatorial design problem. Our proposed model can provide high-quality partition that guarantees that the computing load can be evenly distributed to each worker and minimizes the communication cost with a near-optimal theoretical boundary.Based on the proposed model, we extend the hybrid-cut partitioning algorithm for the power-law graph and propose HCPD, a hybrid-cut partitioning algorithm based on combinatorial design. HCPD uses the proposed model to optimize the load balancing and communication cost simultaneously for high-degree vertices, and assigns the high-degree vertices and their low-degree neighbors to the same workers by label propagation to reduce the overall communication cost. In this way, we partition the low-degree and high-degree vertices holistically and further improve the partition quality, unlike Powerlyra and TopoX, which deal with the two parts independently. Our experiments show that HCPD outperforms Powerlyra on PageRank task by up to 2× faster on real-world power-law graphs with billions of edges.

Recent advances in machine learning-aided lossy compression are incorporating perceptual fidelity into the rate-distortion theory. In this paper, we study the rate-distortion-perception trade-off when the perceptual quality is measured by the total variation distance between the empirical and product distributions of the discrete memoryless source and its reconstruction. We consider the general setting, where two types of resources are available at both the encoder and decoder: a common side information sequence, correlated with the source sequence, and common randomness. We consider both the strong perceptual constraint and the weaker empirical perceptual constraint. The required communication rate for achieving the distortion and empirical perceptual constraint is the minimum conditional mutual information, and similar result holds for strong perceptual constraint when sufficient common randomness is provided and the output along with the side information is constraint to an independent and identically distributed sequence.

This paper presents a low-complexity lossy compression scheme for Gaussian vectors, using sparse regression codes (SRC) and a novel decimated approximate message passing (AMP) encoder. The sparse regression codebook is characterized by a design matrix and each codeword is a linear combination of selected columns of the matrix. In order to enable the convergence of AMP for lossy compression, we incorporate the concept of decimation into the AMP algorithm for the first time. Further, we show that the power allocation technique is beneficial for improving the rate-distortion performance. The computational complexity of the proposed encoding is O(log n) per source sample for a length-n source vector, using a sub-Fourier design matrix. Moreover, the proposed AMP encoder inherently supports successively refinable compression. Simulation results show that the proposed decimated AMP encoder significantly outperforms the existing successive-approximation encoding [1] and approaches the rate-distortion limit in low-rate regime.

SimRank is an important metric to measure the topological similarity between two nodes in a graph. In particular, single-source and top-k SimRank has numerous applications in recommendation systems, network analysis, and web mining, etc. Mathematically, given a vertex, the computation of single-machine and single-source SimRank mainly lies in matrix-matrix operations. However, it is almost impossible to directly compute on large graphs. Thus, existing works yield to two main operations: a series of random walks, and sparse matrix and dense vector multiplication operations. This brings about high computation cost for SimRank on large graphs. In real-world applications, there is always the query time and accuracy trade-off, which hinders the computation of high-precision SimRank on large-scale graphs. To handle this problem, this paper proposesClipSim, the first GPU-friendly parallel framework that accelerates the single-source SimRank on GPU with accuracy guarantee. We design a novel data structure and GPU-friendly parallel algorithms for efficient computation of all the operations of SimRank on GPU. Moreover, our theoretical derivation enables ClipSim to largely reduce the number of random walks required for each node, while maintaining the same theoretical accuracy as the state-of-the-art algorithm, ExactSim. We conduct extensive experiments on real-world and synthetic datasets to demonstrate the accuracy and efficiency of ClipSim. The results show that compared with ExactSim, ClipSim obtains single-source SimRank vectors with the same accuracy and up to 160× faster computation time.

In this paper, we study a concatenate coding scheme based on sparse regression code (SPARC) and tree code for unsourced random access in massive multiple-input and multiple-output systems. Our focus is concentrated on efficient decoding for the inner SPARC with practical concerns. A two-stage method is proposed to achieve near-optimal performance while maintaining low computational complexity. Specifically, a one-step thresholding-based algorithm is first used for reducing large dimensions of the SPARC decoding, after which a relaxed maximum-likelihood estimator is employed for refinement. Adequate simulation results are provided to validate the near-optimal performance and the low computational complexity. Besides, for covariance-based sparse recovery method, theoretical analyses are given to characterize the upper bound of the number of active users supported when convex relaxation is considered, and the probability of successful dimension reduction by the one-step thresholding-based algorithm.

In this paper, we study a concatenate coding scheme based on sparse regression code (SPARC) and tree code for unsourced random access in massive multiple-input and multiple-output systems. Our focus is concentrated on efficient decoding for the inner SPARC with practical concerns. A two-stage method is proposed to achieve near-optimal performance while maintaining low computational complexity [1]. Specifically, a one-step thresholding-based algorithm is first used for reducing large dimensions of the SPARC decoding, after which a relaxed maximum-likelihood estimator is employed for refinement. Adequate simulation results are provided to validate the near-optimal performance and the low computational complexity. Besides, for covariance-based sparse recovery method, theoretical analyses are given to characterize the upper bound of the number of active users supported when convex relaxation is considered, and the probability of successful dimension reduction by the one-step thresholding-based algorithm.

We consider a semantic cipher system, in which we protect only the semantic information of the source. The optimal tradeoff is characterized among the coding rate, the secret key rate, the semantic information leakage rate, the source reconstruction distortion, and the semantic distortion. It is shown that an efficient key with a small size suffices to protect the semantic information.

We study an estimation-compression (EC) separation scheme in a semantic communication system. Therein, the semantic information is intrinsic and not observable. The EC scheme first estimates the semantic information from the observed message and then compresses the estimation subject to a rate-distortion regime. The corresponding EC rate-distortion tradeoff is obtained. In particular, the EC separation scheme achieves the semantic rate-distortion function if the estimation is a sufficient statistic of the semantic information based on the observed message. Moreover, the extra distortion incurred by the compression in addition to the irreducible error in semantic estimation problems is also analyzed. A binary classification of vector Gaussian observations is investigated. We design an optimal soft decision estimator which is a sufficient statistic and show that it strictly outperforms the Bayesian decision estimator in terms of rate-distortion tradeoff. As the dimension of the observed Gaussian vector increases, the performance gap between the Bayesian decision estimator and the soft decision estimator becomes smaller and smaller until it is negligible.

We consider a problem of coding for computing, where the decoder wishes to estimate a function of its local message and the source message at the encoder within a given distortion. We show that the rate-distortion function can be characterized through a characteristic multi-hypergraph, which simplifies the evaluation of the rate-distortion function.

This paper considers a concatenated coding scheme of sparse regression codes (SPARC) and tree code for asynchronous uplink unsourced random access. The encoding of SPARC is limited in a single sub-carrier to counter the unknown delay due to asynchronization. A slotted structure, where both channel uses and potential users are divided into slots, is added to reduce the overall computational complexity and improve its reliability when the length of the coherence block is limited. An efficient two-stage decoder is applied for further computational complexity reduction at the cost of slightly higher per-user probability of error. Asymptotic and numerical results are provided to demonstrate the effectiveness of the proposed scheme.

The accelerated convergence of digital and real-world lifestyles has imposed unprecedented demands on today’s wireless network architectures, as it is highly desirable for such architectures to support wireless devices everywhere with high capacity and minimal signaling overhead. Conventional architectures, such as cellular architectures, are not able to satisfy these requirements simultaneously, and are thus no longer suitable for the future era. In this paper, we propose a capacity-centric (C2) architecture for future wireless networking. It is designed based on the principles of maximizing the number of non-overlapping clusters with the average cluster capacity guaranteed to be higher than a certain threshold, and thus provides a flexible way to balance the capacity requirement against the signaling overhead. Our analytical results reveal that C2 has superior generality, wherein both the cellular and the fully coordinated architectures can be viewed as its extreme cases. Simulation results show that the average capacity of C2 is at least three times higher compared to that of the cellular architecture. More importantly, different from the widely adopted conventional wisdom that base-station distributions dominate architecture designs, we find that the C2 architecture is not over-reliant on base-station distributions, and instead the user-side information plays a vital role and cannot be ignored.

The spectral efficiency of small-cell wireless networks is limited by the backhaul capacity of the base stations (BSs) as well as the severe interference from the neighboring BSs. One promising approach to improve the spectral efficiency of small-cell wireless networks is cache-aided cooperative transmission, where caching at the BSs can alleviate the high-speed backhaul capacity requirement and cooperative transmission can enhance the signal-to-interference-plus-noise ratio. A key issue is that the cached content may not be located at the nearest BS, which means that to access such content, a user needs to overcome strong interference from the nearby BSs. We propose an interference-aware dual-mode caching and user-centric open-loop cooperative transmission scheme that embraces spatial caching diversity and user-centric open-loop cooperative transmission, and alleviate the interference issue in the system. Based on the proposed scheme, we derive a tractable expression of the average successful transmission probability (STP) in terms of key system parameters. We then design a low-complexity algorithm to optimize the average STP with respect to the bandwidth and the cache storage capacity allocation. Simulations show that the proposed scheme achieves a higher average STP than the existing caching schemes.

Caching at base stations (BSs) is a promising approach for supporting the tremendous traffic growth of content delivery over future small-cell wireless networks with limited backhaul. This paper considers exploiting spatial caching diversity (i.e., caching different subsets of popular content files at neighboring BSs) that can greatly improve the cache hit probability, thereby leading to better overall system performance. A key issue in exploiting spatial caching diversity is that the cached content may not be located at the nearest BS, which means that to access such content, a user needs to overcome strong interference from the nearby BSs; this significantly limits the gain of spatial caching diversity. In this paper, we consider a joint design of frequency reuse and caching, such that the benefit of an improved cache hit probability induced by spatial caching diversity and the benefit of interference coordination induced by frequency reuse can be achieved simultaneously. We obtain a closed-form characterization of the approximate successful transmission probability for the proposed scheme and analyze the impact of key operating parameters on the performance. We design a low-complexity algorithm to optimize the frequency reuse factor and the cache storage allocation. Simulations show that the proposed scheme achieves a higher successful transmission probability than existing caching schemes.

In this paper, we study the average sum DoF of 2 cached interference networks over random user requests. First, 3 we derive a closed-form average sum DoF upper bound of 4 the cached interference channels over all possible caching and 5 transmission strategies. Then, we propose an online cache content 6 placement algorithm to maximize the achievable average sum 7 DoF without explicit knowledge of the popularity of the content 8 files. We show that the gap between the DoF upper bound and 9 an achievable DoF is small for some typical scenarios. Moreover, 10 we quantify the DoF gain due to caching with respect to some important system parameters. We show that large DoF gain over 11 12 interference network without cache is possible even if the number 13 of files L → ∞. Both simulation and analysis indicate that the proposed scheme has significant gains over various baselines.

Recently, physical layer (PHY) caching at the base station (BS) has been proposed to induce MIMO cooperation among peer BSs to reduce interference and improve the spectrum efficiency in the radio access network (RAN). Compared with caching at the core network (CN) gateway (CN-caching), which covers a large number of users, it may appear that PHY-caching covers less users and results in lower cache hit rate. In this paper, we analyze the tradeoff between spectrum efficiency gain induced by PHY-caching and CN-backhaul offloading gain induced by CN-caching. Specifically, we derive the Pareto optimal tradeoff between the spectrum efficiency gain and CN-backhaul offloading gain. Surprisingly, we show that even though PHY-caching covers less users than CN-caching (single cell versus multiple cells), it is still worthwhile to do PHY-caching at the BS.

Interference is a key issue that limits the Degrees of Freedom (DoF) in conventional wireless networks. In this paper, we propose a novel physical layer (PHY) caching scheme to achieve DoF gains in interference networks. Specifically, by properly caching a portion of the content files at each transmitters, the PHY topology can be opportunistically transformed from the unfavorable interference channel topology into a more favorable MIMO broadcast channel topology and enjoy a large DoF gain. We first propose a novel caching scheme to significantly improve the MIMO cooperation opportunity induced by PHY caching. Then we quantify the DoF gain w.r.t. some important system parameters. Both analysis and simulation show that the proposed scheme has significant gains over various baselines.

In this paper, we address the problem of distributed angle estimation in wireless sensor localization system. Given that the practical limitations on size and cost, i.e., anchor node cannot equip a complicated antenna or antenna array, we devise an alternative measure based on a fixed-spacing two-antenna equipment. Through the multiple frequencies tuning, a virtual non-uniform linear array, called multiple frequencies synthetic array (MFSA), can be rebuilt independently at each sensor node. We first provide in theory the angle unambiguity conditions for such virtual array and then propose two different algorithms to achieve the distributed angle of departure (AOD) estimation. The first algorithm is based on the spectral searching technique, which is an improved version of the conventional rank reduction estimator (RARE) and has the same performance but less computational complexity. To further alleviate the computational burden at sensor node end, we convert the above angle estimation into a congruence problem and provide another closed-form solution based on Chinese remainder theorem (CRT). We also derive the Cramer-Rao bound for the MFSA and demonstrate theoretically that the latter algorithm from the perspective of cumulative circular distance is suboptimal when compared with the former one. Numerical examples are provided to show the validity of the proposed algorithms and the corresponding conclusions.

In this paper linear precoders for non-cooperative transmitters and iterative multiuser detectors (MUD) for the receiver are jointly designed to achieve the individual quality-of-service (QoS). An evolutional analysis framework is first developed for predicting and tracking the performance evolution for each user along iterations. This semi-analytical tool then leads to a joint QoS and quality-of-convergence (QoC) constrained precoder design problem formulation that is complicated and non-convex. Due to the competitive nature of a multiaccess system, a game-theoretic reformulation is adopted to reveal the underlying partial convexity structure of the problem, which further facilitates the development of an efficient iterative solver for the problem. Extensive simulation results are given to support the mathematical development of the problem.

In this paper linear precoder at the transmitter and iterative frequency domain equalizer (FDE) at the receiver are jointly designed to achieve the target quality-of-service (QoS) with minimum transmission power consumption. We first devise a semi-analytical tool that precisely tracks the performance evolution of the iterative process, and then provide a set of quality-of-convergence (QoC) constraints such that the target QoS can be achieved after iterations. We thus formulate the design problem as a joint QoS/QoC constrained power allocation problem, which is shown to be convex. Despite the widely adopted suboptimal solvers, we develop a specific IPM solver by revealing the underlying sparsity in the mathematical structure of the problem, which significantly reduces the computational cost, and keeps the performance advantages as well. Extensive simulation results are given to support the mathematical development of the problem.

When the arrival timing difference exists in the uplink MUMIMO systems, the conventional MUD techniques, without timing control, failed under this circumstance. In this paper, we proposed a joint distortion precancelation and iterative FDE architecture, without timing control, for the asynchronous system. The proposed method derives the iterative FDE filters, via the MMSE criterion, by treating the precanceled distortion components as colored noise. The asymptomatic MSE performance is shown to approach MFB, and the achievable SER performance is evaluated in various system settings, which confirms its effectiveness.

Single-carrier block transmission systems with frequency domain equalization (FDE) suffers from power and bandwidth efficiency loss due to the cyclic extension. Conventional schemes to reduce the cyclic extension needed rely heavily on Turbo architecture which has much higher computational complexity. To make good compromise between performance and complexity, we proposed a precoding aided iterative FDE technique with reduced-length cyclic extensions. We emphasized that the proposed scheme effectively redistributes the dominant part of residual distortion and ensures reliable symbol estimate, which is superior to the conventional non-Turbo schemes. Low complexity algorithms are presented for computing key parameters and corresponding FDE filters. Simulation results confirmed the effectiveness of the proposed scheme for both uncoded and coded systems, which achieves a bandwidth efficiency increase of 29.4% and a power efficiency gain of 1.12dB, in the Hiperlan2/E system settings.

In this paper, we deal with channel estimation and intercarrier interference mitigation for very high mobility OFDM systems, the communication is between a BS and a train traveling at a speed of 500 km/hr. The scheme proposed here uses piece-wise polynomial expansion to approximate time-variations of multipath channels. Two adjacent symbols are utilized to estimate the first and second-order parameters. In order to improve the estimation accuracy and mitigate the intercarrier interference caused by pilot-aided channel estimation, the multipath channel parameters were re-estimated in time-domain employing the tentative decision of current OFDM symbol. Simulation results show that this method would improve estimation accuracy and system performance in a high-mobility environment.