We present a system for automatic multi-axis perceptual quality prediction of generative audio, developed for Track 2 of the AudioMOS Challenge 2025. The task is to predict four Audio Aesthetic Scores—Production Quality, Production Complexity, Content Enjoyment, and Content Usefulness—for audio generated by text-to-speech (TTS), text-to-audio (TTA), and text-to-music (TTM) systems. A main challenge is the domain shift between natural training data and synthetic evaluation data. To address this, we combine BEATs, a pretrained transformer based audio representation model, with a multi-branch long short-term memory (LSTM) predictor and use a triplet loss with buffer-based sampling to structure the embedding space by perceptual similarity. Our results show that this improves embedding discriminability and generalization, enabling domain robust audio quality assessment without synthetic training data.

Federated learning (FL) is a leading approach for iterative learning using possibly private data available at edge devices. The federated operation gives rise to challenges in privacy leakage, which accumulates in learning, and communication latency. These limitations are often individually mitigated by the introduction of privacy preserving noise and user-selection policies, typically at the cost of accuracy. In this work, we propose Privacy-aware Active User SElection (PAUSE), which balances the trade-off between privacy accumulation, communication latency, and optimization of the learned model, via dedicated user selection. This triplet is used to construct a reward (cost function), according to which a multi-armed bandit (MAB)-based algorithm dynamically chooses a subset of users in each round, while guaranteeing bounded accumulated privacy leakage. We establish a theoretical analysis, systematically showing that the reward growth rate of PAUSE follows the best-known rate in MAB literature. While the privacy guarantees hold by the construction of PAUSE, we numerically validate its associated improved latency and accuracy gains in different experimental settings of FL.

In this paper, we study epidemic detection in a geographical region where a center for disease control (CDC) relies on two distinct testing agencies to assess an outbreak. Each agency operates within a defined area, and the quality of their testing performance can vary, leading to missed detections or false negatives. The CDC observes the test results from each agency and must detect whether an outbreak is occurring (or not). The CDC’s role is to perform distributed spatial sampling, i.e., define specific regions tested by each agency to optimize the collective detection error and enhance the reliability of epidemic detection. We refer to this decision-making challenge as the CDC problem. In this work, we focus on the CDC problem under the assumptions that (i) the epidemic is modeled as a spatially homogeneous Poisson counting process, and (ii) testing results are only affected by missed detections (without considering false positives). For this setting, we analyze how the distributed spatial sampling strategies (which may comprise disjointed or partially overlapping regions) of the testing agencies influence the overall detection accuracy. We derive optimal coverage strategies for each agency (and hence, for the CDC), with the objective of minimizing detection error. Notably, we demonstrate that the optimal error exponent can be expressed as a simple optimization problem, which we solve completely for the case of two testing agencies.

Graph representations of data offer a rich framework for advanced signal processing applications. However, in many practical scenarios, constructing the graph is computationally demanding, and storing it can be prohibitively expensive-often requiring significantly more memory than the signal itself. This paper introducesGraphLite, a novel algorithm tailored for one of the good performing graph learning methods, to compactly represent the learned graph through an auxiliary vector of the same dimension as the signal. This auxiliary representation, which consists of the inverse node degree profile, arises naturally from the structure of the optimal solution and can be pre-computed and stored alongside the signal. The result is a lightweight, lossless graph representation that retains compatibility with core graph signal processing (GSP) operations. GraphLite offers a flexible and memory-efficient tool for downstream tasks in applications where both online graph construction and storage are bottlenecks.

The A-channel is a noiseless multiple access channel in which users simultaneously transmitQ−ary symbols and the receiver observes the union of all input symbols. An A-channel is said to be unsourced if additionally, all users’ transmissions are encoded across time using a common codebook and decoding is performed without regard to the identities of the active users. Whereas the A-channel employs a traditional set union, the B-channel employs a multiset union so that the receiver observes the set of input symbols together with their respective multiplicities. In this paper, we consider the task of coding for the unsourced A- and B-channels in the presence of i.i.d. erasures and we propose a novel tail-biting code called the linked loop code (LLC) for these channels. The LLC is shown to outperform contemporary codes in part due to its resilience to lost sections. The performance of the LLC code is investigated theoretically and bounds on the error performance are provided.

The Rényi Information Dimension (RID) is a fundamental measure for quantifying the compressibility of random variables with singularities in their distributions, extending beyond classical notions of sparsity. At a high level, RID represents the average number of bits required to encode i.i.d. samples of a random variable with high precision. For stochastic processes, two main extensions of RID exist: the information dimension rate (IDR) and the block information dimension (BID). A more recent approach to characterizing the compressibility of stochastic processes is through ϵ-achievable compression rates, which treat a random process as the limit of finite-dimensional random vectors and leverage tools from compressed sensing. However, the interplay between BID, IDR, and ϵ-achievable compression rates remains poorly understood. Furthermore, explicit values of IDR and BID are known only for a limited class of processes, such as i.i.d. sequences (i.e., discrete-time white noise) and moving-average (MA) processes. This paper investigates the IDR and BID of discrete-time Auto-Regressive Moving-Average (ARMA) processes and their relationship with ϵ-achievable compression rates when the excitation noise follows a discrete-continuous distribution. Specifically, we show that the RID and ϵ-achievable compression rates of such ARMA processes are equal to those of their excitation noise. In other words, despite the fact that ARMA process samples are not sparse, their compressibility matches that of their sparse excitation noise. To establish this result, we demonstrate that the singular components of the sample distribution are supported on affine sets, with relative dimensions that concentrate around the BID. Leveraging a known result on typical affinely singular sources, we further prove that in this setting, the RID coincides with ϵ-achievable compression rates. The findings of this paper provide new insights into the compressibility of locally correlated data with finite- or infinite-memory, which are commonly modeled using ARMA processes.

This paper introduces the Selection Network Assisted Personalized CSI Compression (SNAP-CSI) algorithm, a novel approach for efficient Channel State Information (CSI) compression in wireless networks. Focusing on scenarios where CSI from User Equipment (UE) is transmitted to a Base Station (BS) via a rate-limited channel, SNAP-CSI employs Deep Neural Networks (DNNs) trained on historical CSI data for enhanced compression. Central to SNAP-CSI is the exploitation of CSI heterogeneity to cluster users, enabling the training of tailored personalized models. These models comprise an encoder at the UE and a decoder at the BS, optimized for efficient compression with minimal parameters, specific to each user cluster. A key innovation in SNAP-CSI is the development of a Selection Network (SN). This network predicts cluster membership from compressed CSI data, allowing UEs to select the most fitting personalized model for the lowest data distortion. Concurrently, the BS utilizes the SN for accurate reconstruction of compressed CSI, thus negating the need for further synchronization. The effectiveness of SNAP-CSI is validated through simulations with the ultra-dense indoor maMIMO dataset, evaluating performance across diverse heterogeneity conditions and UE-to-BS channel rates.

Time-encoded sampling represents an emerging paradigm for temporal discretization, garnering recent interest. In this paper, we address the challenge of time-encoded sampling for the perfect recovery of signals residing in shift-invariant spaces (SISs) defined by multiple generators. Specifically, we establish a sufficient condition for achieving perfect recovery of a signal from a multigenerator SIS when it is sampled using a single Time-Encoded Machine (TEM). This condition is exclusively characterized in terms of the provided set of multiple generators. Our findings extend the prior work of Gontier and Vetterli [ACHA, ’13], which originally focused on single-generator SISs. We present illustrative results for both classic baseband signals and bandpass signals (both examples of SISs), highlighting the limitations of single-TEM-based time-encoding schemes.

In [1], the linked loop code (LLC) is presented as a promising code for the unsourced A-channel with erasures (UACE). The UACE is an unsourced multiple access channel in which active users’ transmitted symbols are erased with a given probability and the channel output is obtained as the union of the non-erased symbols. In this paper, we extend the UACE channel model to the unsourced B-channel with erasures (UBCE). The UBCE differs from the UACE in that the channel output is the multiset union – or bag union– of the non-erased input symbols. In other words, the UBCE preserves the symbol multiplicity of the channel output while the UACE does not. Both the UACE and UBCE find applications in modeling aspects of unsourced random access. The LLC from [1] is enhanced and shown to outperform the tree code over the UBCE. Findings are supported by numerical simulations.

In this work, we study the problem of distributed sampling and interpolation for perfect reconstruction of graph signals. In particular, we explore and present a generalization of Papoulis' classic generalized sampling expansion (GSE) to graph signals. We consider a single-time instance of a graph signal from a space of bandlimited graph signals, appropriately defined via the graph Fourier transform associated to the graph. For such bandlimited graph signals, we first identify a sufficient condition for perfect reconstruction via distributed sampler/interpolator pairs, in the spirit of the Shannon-Nyquist criterion. When this perfect reconstruction criteria is satisfied by the individual sampler rates, we then propose a distributed sampler/interpolator architecture which is shown to be achievable for the underlying bandlimited space. The results represent a unique generalization of Papoulis' generalized sampling expansion (GSE) paradigm to graph signals. Interestingly, our results show that such achievable schemes-comprising several pairs of individual sampler/interpolator pairs- are such that every component sampler can be essentially perceived as a concatenation of a pre-sampling filtering operation followed by binary vertex-sampling. The corresponding interpolator is then obtained as a linear transformation which is completely dependent on the vertex-sampling operation but is independent of the pre-sampling filter.

This paper considers the problem of hypothesis testing through the distributed sampling of a remote Poisson source. More specifically, we consider the scenario in which one of two Poisson sources is observed at a set of$K$remote observers. These source observations are subject to erasure-type noise, so some of the source spikes are not received at some of the$K$observers, leading to incomplete signal reception. The partially received signal is then transmitted to a central detector whose task is to identify the originating source. A crucial constraint in our study is the limited capacity for signal forwarding from the observers to the central detector. We assume that these remote observers are subject to a sampling constraint so that only a portion of the total signal received at all remote observers can be forwarded to the central detector. Given this sampling constraint, we determine the optimal sampling strategy at the remote observers that minimizes the probability of error in the detection of the remote source. This problem setting is motivated by the problem of testing of large populations through multiple tests, each subject to a certain false positive and false negative rates. Our paper contributes to the field through a comprehensive mathematical analysis, providing innovative strategies and insights for efficient resource allocation in large-scale testing. The proposed model not only enhances understanding of distributed Poisson sampling under constraints but also offers practical applications in robust decision-making for hypothesis testing in complex environments.

In the federated training of a deep neural network (DNN), model updates are transmitted from the remote users to the parameter server (PS). In many scenarios of practical relevance, one is interested in reducing the communication overhead to enhance training efficiency. To address this challenge, we introduce$\textsf {CO}_{3}$.$\textsf {CO}_{3}$takes its name from three processing applied which reduce the communication load when transmitting the local DNN gradients from the remote users to the PS. Namely, 1) gradient quantization through floating-point conversion; 2) lossless compression of the quantized gradient; and 3) correction of quantization error. We carefully design each of the steps above to ensure good training performance under a constraint on the communication rate. In particular, in steps 1) and 2), we adopt the assumption that DNN gradients are distributed according to a generalized normal distribution, which is validated numerically in this article. For step 3), we utilize an error feedback with a memory decay mechanism to correct the quantization error introduced in step 1). We argue that the memory decay coefficient –similar to the learning rate—can be optimally tuned to improve convergence. A rigorous convergence analysis of the proposed$\textsf {CO}_{3}$with stochastic gradient descent (SGD) is provided. Moreover, with extensive simulations, we show that$\textsf {CO}_{3}$offers improved performance as compared with existing gradient compression schemes proposed in the literature which employ sketching and nonuniform quantization of the local gradients.

In federated learning (FL), the communication constraint between the remote clients and the Parameter Server (PS) is a crucial bottleneck. For this reason, model updates must be compressed so as to minimize the loss in accuracy resulting from the communication constraint. This paper proposes “M-magnitude weighted L2 distortion + 2 degrees of freedom” (M22) algorithm, a rate-distortion inspired approach to gradient compression for federated training of deep neural networks (DNNs). In particular, we propose a family of distortion measures between the original gradient and the reconstruction we referred to as “$M$-magnitude weighted$L_{2}$” distortion, and we assume that gradient updates follow an i.i.d. distribution – generalized normal or Weibull, which have two degrees of freedom. In both the distortion measure and the gradient distribution, there is one free parameter for each that can be fitted as a function of the iteration number. Given a choice of gradient distribution and distortion measure, we design the quantizer to minimize the expected distortion in gradient reconstruction. To measure the gradient compression performance under a communication constraint, we define the per-bit accuracy as the optimal improvement in accuracy that one bit of communication brings to the centralized model over the training period. Using this performance measure, we systematically benchmark the choice of gradient distribution and distortion measure. We provide substantial insights on the role of these choices and argue that significant performance improvements can be attained using such a rate-distortion inspired compressor.

The empirical risk minimization (ERM) problem with relative entropy regularization (ERM-RER) is investigated under the assumption that the reference measure is a σ-finite measure, and not necessarily a probability measure. Under this assumption, which leads to a generalization of the ERM-RER problem allowing a larger degree of flexibility for incorporating prior knowledge, numerous relevant properties are stated. Among these properties, the solution to this problem, if it exists, is shown to be a unique probability measure, mutually absolutely continuous with the reference measure. Such a solution exhibits a probably-approximately-correct guarantee for the ERM problem independently of whether the latter possesses a solution. For a fixed dataset and under a specific condition, the empirical risk is shown to be a sub-Gaussian random variable when the models are sampled from the solution to the ERM-RER problem. The generalization capabilities of the solution to the ERM-RER problem (the Gibbs algorithm) are studied via the sensitivity of the expected empirical risk to deviations from such a solution towards alternative probability measures. Finally, an interesting connection between sensitivity, generalization error, and lautum information is established.

In federated learning (FL), the communication constraint between the remote users and the Parameter Server (PS) is a crucial bottleneck. This paper proposes M22, a rate-distortion inspired approach to model update compression for distributed training of deep neural networks (DNNs). In particular, (i) we propose a family of distortion measures referred to as "M-magnitude weighted L2" norm, and (ii) we assume that gradient updates follow an i.i.d. distribution with two degrees of freedom – generalized normal and Weibull distributions. To measure the gradient compression performance under a communication constraint, we define the per-bit accuracy as the optimal improvement in accuracy that a bit of communication brings to the centralized model over the training period. Using this performance measure, we systematically benchmark the choice of gradient distributions and the distortion measure. We provide substantial insights on the role of these choices and argue that significant performance improvements can be attained using such a rate-distortion inspired compressor.

We study the problem of distributed sampling and detection of remote point processes. A remote source, modelled as a homogeneous Poisson counting process (PCP) is observed at multiple remote observers in noise. The observers have a sampling constraint which limits their ability to forward their observations to a centralized fusion center, or ‘detector’. More precisely, we assume that the remote observers can send any fixed fraction of their observation to the detector noiselessly; in addition, the overall time duration of the observation received, or the ON time, at the detector is limited. We refer to this constraint as a joint sampling/communication constraint, as it accounts for both of the following: (i) the finite energy available for sampling at the remote observers, and (ii) the finite capacity of the uplink toward the fusion center. Our main contribution is the complete characterization of optimal strategies for joint sampling and detection of the remote source. We first present optimal strategies when there are two samplers, and then extend the characterization for the K-sampler, K > 2, case. Our results reveal a fundamental tension in the design of distributed sampling strategies between (i) obtaining noisy observations of the remote source at multiple samplers so as to jointly ‘reject’ their individual observation noise at the detector, and (ii) observing noisy realization at exactly one appropriately chosen sampler over a longer time period to obtain a better estimate of the remote PCP source intensity. Our results also reveal the interesting fact that two simultaneously active samplers are necessary and sufficient for complete noise-rejection.

Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks. This approach is referred to as (). s differ from other approaches in the literature as they are data-driven estimators. As such, they have the potential to perform well on a large class of capacity problems. To test the performance across various s, it is desirable to establish a benchmark encompassing the different challenges of capacity estimation. This is the objective of this paper. We consider three scenarios for benchmarking: (i) the classic AWGN channel, (ii) channels continuous inputs– the optical intensity and peak-power constrained AWGN channel (iii) channels with a discrete output– i.e., the Poisson channel. We also consider the extension to the multi-terminal case with (iv) the AWGN and optical MAC models. We argue that benchmarking a certain across these four scenarios provides a substantive test of performance. We study the performance ofmutual information neural estimator(MINE),smoothed mutual information lower-bound estimator(SMILE), anddirected information neural estimator(DINE) and provide insights into the performance of other methods as well. To summarize our benchmarking results, MINE provides the most reliable performance.

In this paper, we study the problem of estimating the direction of arrival (DOA) using a sparsely sampled uniform linear array (ULA). Based on an initial incomplete ULA measurements, our strategy is to choose a sparse subset of array elements for measuring the next snapshot. Then, we use a Hankel-structured matrix completion to interpolate for the missing ULA measurements. Finally, the source DOAs are estimated using a subspace method such as Prony on the fully recovered ULA. We theoretically provide a sufficient bound for the number of required samples (array elements) for perfect recovery. The numerical comparisons of the proposed method with existing techniques such as atomic-norm minimization and off-the-grid approaches confirm the superiority of the proposed method.

In this paper, the problem of optimal gradient lossless compression in Deep Neural Network (DNN) training is considered. Gradient compression is relevant in many distributed DNN training scenarios, including the recently popular federated learning (FL) scenario in which each remote users are connected to the parameter server (PS) through a noiseless but rate limited channel. In distributed DNN training, if the underlying gradient distribution is available, classical lossless compression approaches can be used to reduce the number of bits required for communicating the gradient entries. Mean field analysis has suggested that gradient updates can be considered as independent random variables, while Laplace approximation can be used to argue that gradient has a distribution approximating the normal (Norm) distribution in some regimes. In this paper we argue that, for some networks of practical interest, the gradient entries can be well modelled as having a generalized normal (GenNorm) distribution. We provide numerical evaluations to validate that the hypothesis GenNorm modelling provides a more accurate prediction of the DNN gradient tail distribution. Additionally, this modeling choice provides concrete improvement in terms of lossless compression of the gradients when applying classical fix-to-variable lossless coding algorithms, such as Huffman coding, to the quantized gradient updates. This latter results indeed provides an effective compression strategy with low memory and computational complexity that has great practical relevance in distributed DNN training scenarios.

Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks and without the knowledge of closed form distribution of the data. This class of estimators is referred to as neural mutual information estimators. Although very promising, such techniques have yet to be rigorously bench-marked so as to establish their efficacy, ease of implementation, and stability for capacity estimation which is joint maximization frame-work. In this paper, we compare the different techniques proposed in the literature for estimating capacity and provide a practitioner perspective on their effectiveness. In particular, we study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE) and provide insights on InfoNCE. We evaluated these algorithms in terms of their ability to learn the input distributions that are capacity-approaching for the AWGN channel, the optical intensity channel, and peak power-constrained AWGN channel. For both scenarios, we provide insightful comments on various aspects of the training process, such as accuracy, stability, and sensitivity to initialization.

The optimality and sensitivity of the empirical risk minimization problem with relative entropy regularization (ERM-RER) are investigated for the case in which the reference is a σ-finite measure instead of a probability measure. This generalization allows for a larger degree of flexibility in the incorporation of prior knowledge over the set of models. In this setting, the interplay of the regularization parameter, the reference measure, the risk function, and the empirical risk induced by the solution of the ERM-RER problem is characterized. This characterization yields necessary and sufficient conditions for the existence of regularization parameters that achieve arbitrarily small empirical risk with arbitrarily high probability. Additionally, the sensitivity of the expected empirical risk to deviations from the solution of the ERM-RER problem is studied. Dataset-dependent and dataset-independent upper bounds on the absolute value of the sensitivity are presented. In a special case, it is shown that the expectation (with respect to the datasets) of the absolute value of the sensitivity is upper bounded, up to a constant factor, by the square root of the lautum information between the models and the datasets.

In this paper, we study the detection of Poisson point sources when the central detector observes the remote source via a restricted number of samples from distributed sensors. More specifically, we consider the scenario in which a Poisson source is observed, in noise, at two remote observers or samplers. At each sampler, the noisy observations are sampled as part of a distributed strategy designed by the central detector by accounting for a communication constraint between the sensor and the detector. Such limited sampling/estimation/communication scenarios are fundamental to modern cyberphysical systems. In such systems, discrete events such as signals for detection, control, and feedback propagate through a common communication and sensing infrastructure. For this scenario, we study the problem of optimally selecting the distributed sampling strategy employed at all remote samplers under the constraint that samples can be acquired for a given fraction of time across both samplers. We focus on point processes in this work and derive an optimal sampling strategy for the case of a homogeneous Poisson source which may be corrupted by another independent, additive and homogeneous Poisson noise source with known intensity. We show that any optimal solution combines either or both of these two distributed sampling strategies: (i) a time-sharing strategy –samplers communicate samples corresponding to non-overlapping time intervals, and (ii) a noise rejection strategy –samplers communicate samples during an identical time interval of activity, thus allowing for identification and subsequent rejection of the spurious additive noise realizations at either sampler. We argue that these two strategies play a crucial role in more general scenarios, encompassing a more general class of sources and noise realizations.

In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M2loss between the outputs of the target and of the compressed network, under the assumption of Gaussian inputs. By using tools from high-dimensional probability, we show that this non-convex problem can be simplified when the target network is sufficiently over-parameterized, and provide the error rate of this approximation as a function of the input dimension and N. In this mean-field limit, the simplified objective, as well as the optimal weights of the compressed network, does not depend on the realization of the target network, but only on expected scaling factors. Furthermore, for networks with ReLU activation, we conjecture that the optimum of the simplified optimization problem is achieved by taking weights on the Equiangular Tight Frame (ETF), while the scaling of the weights and the orientation of the ETF depend on the parameters of the target network. Numerical evidence is provided to support this conjecture.

In decentralized optimization, multiple nodes in a network collaborate to minimize the sum of their local loss functions. The information exchange between nodes required for this task, is often limited by network connectivity. We consider a setting in which communication between nodes is hindered by both (i) a finite rate-constraint on the signal transmitted by any node, and (ii) additive noise corrupting the signal received by any node. We propose a novel algorithm for this scenario: Decentralized Lazy Mirror Descent with Differential Exchanges (DLMD-DiffEx), which guarantees convergence of the local estimates to the optimal solution under the given communication constraints. A salient feature of DLMD-DiffEx is the introduction of additional proxy variables that are maintained by the nodes to account for the disagreement in their estimates due to channel noise and rate-constraints. Convergence to the optimal solution is attained by having nodes iteratively exchange these disagreement terms until consensus is achieved. In order to prevent noise accumulation during this exchange, DLMD-DiffEx relies on two sequences: one controlling the power of the transmitted signal, and the other determining the consensus rate. We provide insights on the design of these two sequences which highlights the interplay between consensus rate and noise amplification. We investigate the performance of DLMD-DiffEx both from a theoretical perspective as well as through numerical evaluations on synthetic data and MNIST. MATLAB and Python implementations can be found athttps://github.com/rajarshisaha95/DLMD-DiffEx.

Large-scale machine learning and data mining methods routinely distribute computations across multiple agents to parallelize processing. The time required for the computations at the agents is affected by the availability of local resources and/or poor channel conditions, thus giving rise to the “straggler problem.” In this paper, we address this problem for distributed approximate matrix multiplication. In particular, we employ Unequal Error Protection (UEP) codes to obtain an approximation of the matrix product to provide higher protection for the blocks with a higher effect on the multiplication outcome. We characterize the performance of the proposed approach from a theoretical perspective by bounding the expected reconstruction error for matrices with uncorrelated entries. We also apply the proposed coding strategy to the computation of the back-propagation step in the training of a Deep Neural Network (DNN) for an image classification task in the evaluation of the gradients. Our numerical experiments show that it is indeed possible to obtain significant improvements in the overall time required to achieve DNN training convergence by producing approximation of matrix products using UEP codes in the presence of stragglers.

The notion of compressibility of a random measure is a rather general concept which find applications in many contexts from data compression, to signal quantization, and parameter estimation. While compressibility for discrete and continuous measures is generally well understood, the case of discrete-continuous measures is quite subtle. In this paper, we focus on a class of multi-dimensional random measures that have singularities on affine lower-dimensional subsets. We refer to this class of random variables asaffinely singular. Affinely singular random vectors naturally arises when considering linear transformation of component-wise independent discrete-continuous random variables. To measure the compressibility of such distributions, we introduce the new notion of dimensional-rate bias (DRB) which is closely related to the entropy and differential entropy in discrete and continuous cases, respectively. Similar to entropy and differential entropy, DRB is useful in evaluating the mutual information between distributions of the aforementioned type. Besides the DRB, we also evaluate the the RID of these distributions. We further provide an upper-bound for the RID of multi-dimensional random measures that are obtained by Lipschitz functions of component-wise independent discrete-continuous random variables (X). The upper-bound is shown to be achievable when the Lipschitz function is$A \mathrm {X}$, where$A$satisfies${\mathrm{ SPARK}}({A_{m\times n}}) = m+1$(e.g., Vandermonde matrices). When considering discrete-domain moving-average processes with non-Gaussian excitation noise, the above results allow us to evaluate the block-average RID and DRB, as well as to determine a relationship between these parameters and other existing compressibility measures.

This paper investigates the role of dimensionality reduction in efficient communication and differential privacy (DP) of the local datasets at the remote users for over-the-air computation (AirComp)-based federated learning (FL) model. More precisely, we consider the FL setting in which clients are prompted to train a machine learning model by simultaneous channel-aware and limited communications with a parameter server (PS) over a Gaussian multiple-access channel (GMAC), so that transmissions sum coherently at the PS globally aware of the channel coefficients. For this setting, an algorithm is proposed based on applying (i) federated stochastic gradient descent (FedSGD) for training the minimum of a given loss function based on the local gradients, (ii) Johnson-Lindenstrauss (JL) random projection for reducing the dimension of the local updates and (iii) artificial noise to further aid user's privacy. For this scheme, our results show that the local DP (LDP) performance is mainly improved due to injecting noise of greater variance on each dimension while keeping the sensitivity of the projected vectors unchanged. This is while the convergence rate is slowed down compared to the case without dimensionality reduction. As the performance outweighs for the slower convergence, the trade-off between privacy and convergence is higher but is shown to lessen in high-dimensional regime yielding almost the same trade-off with much less communication cost.

In decentralized optimization, multiple nodes in a network collaborate to minimize the sum of their local loss functions. The information exchange between nodes required for this task is often limited by network connectivity. We consider a generalization of this setting, in which communication is further hindered by (i) a finite data-rate constraint on the signal transmitted by any node, and (ii) an additive noise corrupting the signal received by any node. We develop a novel algorithm for this scenario: Decentralized Lazy Mirror Descent with Differential Exchanges (DLMD-DiffEx), which guarantees convergence of the local estimates to the optimal solution. A salient feature of DLMD-DiffEx is the introduction of additional proxy variables that are maintained by the nodes to account for the disagreement in their estimates due to channel noise and data-rate constraints. We investigate the performance of DLMD-DiffEx both from a theoretical perspective as well as through numerical evaluations.

Unsourced random access (URA) is a recently proposed communication paradigm attuned to machine-driven data transfers. In the original URA formulation, all the active devices share the same number of bits per packet. The scenario where several classes of devices transmit concurrently has so far received little attention. An initial solution to this problem takes the form of group successive interference cancellation, where codewords from a class of devices with more resources are recovered first, followed by the decoding of the remaining messages. This article introduces a joint iterative decoding approach rooted in approximate message passing. This framework has a concatenated coding structure borrowed from the single-class coded compressed sensing and admits a solution that offers performance improvement at little added computational complexity. Our findings point to new connections between multiclass URA and compressive demixing. The performance of the envisioned algorithm is validated through numerical simulations.

Consider the problem of estimating a latent signal from a lossy compressed version of the data when the compressor is agnostic to the relation between the signal and the data. This situation arises in a host of modern applications when data is transmitted or stored prior to determining the downstream inference task. Given a bitrate constraint and a distortion measure between the data and its compressed version, let us consider the joint distribution achieving Shannon's rate-distortion (RD) function. Given an estimator and a loss function associated with the downstream inference task, define the RD risk as the expected loss under the RD-achieving distribution. We provide general conditions under which the operational risk in estimating from the compressed data is asymptotically equivalent to the RD risk. The main theoretical tools to prove this equivalence are transportation-cost inequalities in conjunction with properties of compression codes achieving Shannon's RD function. Whenever such equivalence holds, a recipe for designing estimators from datasets undergoing lossy compression without specifying the actual compression technique emerges: design the estimator to minimize the RD risk. Our conditions are simplified in the special cases of discrete memoryless or multivariate normal data. For these scenarios, we derive explicit expressions for the RD risk of several estimators and compare them to the optimal source coding performance associated with full knowledge of the relation between the latent signal and the data.

We consider multiple devices with local datasets collaboratively learning a global model through device-to-device (D2D) communications. The conventional decentralized stochastic gradient descent (DSGD) solution for this problem assumes error-free orthogonal links among the devices. This is based on the assumption of an underlying communication protocol that takes care of the noise, fading, and interference in the wireless medium. In this work, we show the suboptimality of this approach by designing the communication and learning protocols jointly. We first consider a point-to-point (P2P) communication scheme by scheduling D2D transmissions in an orthogonal fashion to minimize interference. Then, we propose a novel over-the-air consensus scheme by exploiting the signal superposition property of wireless transmission, rather than avoiding interference. In the proposed OAC-MAC scheme, multiple nodes align their transmissions toward a single receiver node. For both schemes, we cast the scheduling problem as a graph coloring problem. We then numerically compare the two approaches for the distributed MNIST image classification task under various network conditions. We show that the OAC-MAC scheme attains better convergence speed and final accuracy thanks to the improved robustness against channel fading and noise. We also introduce a noise-aware version of the OAC-MAC scheme with further improvements in the convergence speed and accuracy.

The Renyi's information dimension (RID) of an n-dimensional random vector (RV) is the average dimension of the vector when accounting for non-zero probability measures over lower-dimensional subsets. From an information-theoretical perspective, the RID can be interpreted as a measure of compressibility of a probability distribution. While the RID for continuous and discrete measures is well understood, the case of a discrete-continuous measures presents a number of interesting subtleties. In this paper, we investigate the RID for a class of multi-dimensional discrete-continuous random measures with singularities on affine lower dimensional subsets. This class of RVs, which we term affinely singular, arises from linear transformation of orthogonally singular RVs, that include RVs with singularities on affine subsets parallel to principal axes. We obtain the RID of affinely singular RVs and derive an upper bound for the RID of Lipschitz functions of orthogonally singular RVs. As an application of our results, we consider the example of a moving-average stochastic process with discrete-continuous excitation noise and obtain the RID for samples of this process. We also provide insight about the relationship between the block-average information dimension of the truncated samples, the minimum achievable compression rate, and other measures of compressibility for this process.

Federated learning (FL) refers to the adaptation of a central model based on data sets available at multiple remote users. Two of the common challenges encountered in FL are the fact that training sets obtained by different users are commonly heterogeneous, i.e., arise from different sample distributions, and the need to communicate large amounts of data between the users and the central server over the typically expensive up-link channel. In this work we formulate the problem of FL in which different clusters of users observe labeled samples drawn from different distributions, while operating under constraints on the communication overhead. For such settings, we identify that the combination of statistical heterogeneity and communication constraints induces a tradeoff between the ability of the users of each cluster to learn a proper model and the accuracy in aggregating these models into a global inference rule. We propose an algorithm based on multi-source adaptation methods for such communication-aware clustered FL scenarios which allows to balance these performance measures, and demonstrate its ability to achieve improved inference over conventional federated averaging without inducing additional communication overhead.

In this paper, the sum secure degrees of freedom (SDoF) of the K-user Multiple Input/Single Output (MISO) Broadcast Channel with Confidential Messages (BCCM) and alternating Channel State Information at the Transmitter (CSIT) is investigated. In the MISO BCCM, a K-antenna transmitter (TX) communicates toward K single-antenna receivers (RXs), so that message for RX k is kept secret from RX j with jsum= (2K − 1)/2. Interestingly, this SDoFsumis attained by a rather simple achievability in which the TX uses artificial noise to prevent the decoding of the message of the unintended receivers at RX 1. The proof for the case K = 3 is discussed in detail.

In this paper, a distributed convex optimization algorithm, termed distributed coordinate dual averaging (DCDA) algorithm, is proposed. The DCDA algorithm addresses the scenario of a large distributed optimization problem with limited communication among nodes in the network. Currently known distributed subgradient descent methods, such as the distributed dual averaging or the distributed alternating direction method of multipliers, assume that nodes can exchange messages of large cardinality. Such an assumption on the network communication capabilities is not valid in many scenarios of practical relevance. To address this setting, we propose the DCDA algorithm as a distributed convex optimization algorithm in which the communication between nodes in each round is restricted to a fixed number of dimensions. We bound the rate of convergence under different communication protocols and network architectures for this algorithm. We also consider the extensions to the cases of imperfect gradient knowledge and when transmitted messages are corrupted by additive noise or are quantized. Numerical simulations demonstrating the performance of DCDA in these different settings are also provided.

The joint design of input constellation and low-density parity-check (LDPC) codes to approach the symmetric capacity of the two-user Gaussian multiple access channel is studied. More specifically, multilevel coding is employed at each user to construct a high-order input constellation and the constellations of the users are jointly designed so as to maximize the multiuser shaping gain. At the receiver, each layer of the multilevel coding is jointly decoded among users, while successive cancellation is employed across layers. The LDPC code employed by each user in each layer is designed using EXIT charts to support joint decoding among users for the prescribed per-layer rate and SNR. Numerical simulations are provided to validate the proposed constellation and LDPC code designs.

The discrete-time Wiener phase noise channel with an integrate-and-dump multi-sample receiver, referred to as the Oversampled Wiener Phase Noise (OWPN) channel is studied. The capacity of this channel is characterized by three parameters: transmit power P, oversampling factor L, and the variance of the Wiener phase noise σ2. The capacity of this model is connected to the capacity of two related models: the Oversampled Non-Coherent (ONC) channel and the Additive White Gaussian Noise (AWGN) channel. More precisely, it is shown that (i) the capacity of the OWPN is close to the ONC channel when σ2> L while (ii) the capacity of the OWPN is close to that of the AWGN channel when σ2-1.

We consider the multiterminal remote source coding problem of estimating a Gaussian signal from a bit-restricted representation of distributed linear measurements corrupted by additive white Gaussian noise. For this problem, we study the performance of the multiterminal compress-and-estimate (CE) coding scheme in which multiple remote encoders compress their measurements so as to minimize a local distortion measure which depends solely on the distribution of these measurements. In reconstruction, the decoder estimates the signal from the lossy-compressed measurements having full knowledge of the statistics of the source signal and the noisy measurements. The CE coding scheme is motivated by the scenario in which source encoders, due to their limited capabilities, operate according to a pre-determined compression strategy and cannot adapt to the sensing environment while the fusion center has full knowledge and computational capabilities. We focus, in particular, on two scenarios: the centralized observation model in which measurements are collected at a single remote encoder and the distributed observation model where measurements are provided to multiple remote sensors. In both scenarios, we investigate the performance attainable through the CE coding scheme in which the measurements are compressed according to a quadratic distortion measure and compare it to the performance of the coding scheme having full system knowledge.

The capacity of a Multiple-Input Multiple-Output (MIMO) channel in which the antenna outputs are processed by an analog linear combining network and quantized by a set of threshold quantizers is studied. The linear combining weights and quantization thresholds are selected from a set of possible configurations as a function of the channel matrix. The possible configurations of the combining network model specific analog receiver architectures, such as single antenna selection, sign quantization of the antenna outputs or linear processing of the outputs. An interesting connection between the capacity of this channel and a constrained sphere packing problem in which unit spheres are packed in a hyperplane arrangement is shown. From a high-level perspective, this follows from the fact that each threshold quantizer can be viewed as a hyperplane partitioning the transmitter signal space. Accordingly, the output of the set of quantizers corresponds to the possible regions induced by the hyperplane arrangement corresponding to the channel realization and receiver configuration. This connection provides a number of important insights into the design of quantization architectures for MIMO receivers; for instance, it shows that for a given number of quantizers, choosing configurations which induce a larger number of partitions can lead to higher rates1.

The degrees of freedom of a class of discrete-time non-coherent channels with oversampling, termed the Oversampled Non-Coherent (ONC) channel, are shown. The ONC channel is obtained from the classic continuous-time AWGN channel by considering the scenario in which the channel output is also corrupted by phase noise (PN) and processed by a multi-sample receiver. The continuous-time PN process has high variability which results in receiver output samples affected by a discrete-time PN process iid uniformly distributed over the unit circle. The ONC channel models the non-coherent detection scenario in which oversampling is employed for phase recovery but the PN process has such a high variance that its samples appear independent and uniformly distributed despite the oversampling. As such, the assumption of independent and uniformly distributed discrete PN is a limiting assumption that, generally speaking, well approximates the scenario in which the PN coherence time is much smaller than the oversampling time. In this paper, we obtain the generalized degrees of freedom for the case in which the oversampling factor L grows with the transmit power P as Pα. Perhaps surprisingly, we show that no degree of freedom for reliable information transfer is available for L > P2. We conjecture that the same capacity asymptotic holds for other PN channels with oversampling, such as the oversampled Wiener PN channel, when the noise variance grows to infinity faster than the sampling rate.

The Writing onto Fast Fading Dirt (WFFD) channel is investigated to study the effect of partial channel knowledge on the performance of interference pre-cancellation. The WFFD channel is the Gel'fand-Pinsker channel in which the channel output is the sum of the channel input, white Gaussian noise, and a fading-times-state term. The fading-times-state term is obtained as the product of the channel state sequence, known only at the transmitter, and a fast fading process, known only at the receiver. We consider the case of Gaussian-distributed channel states and derive an approximate characterization of capacity for different classes of fading distributions, both continuous and discrete. In particular, we prove that if the fading distribution concentrates in a sufficiently small interval, then capacity is approximately equal to the AWGN capacity times the probability of such interval. We also show that there exists a class of fading distributions for which having the transmitter treat the fading-times-state term as additional noise closely approaches capacity.

The capacity of the discrete-time channel affected by both additive Gaussian noise and Wiener phase noise is studied. Novel inner and outer bounds are presented, which differ of at most 7.36 bit-per-channel-use for all channel parameters. The capacity of this model can be subdivided in three regimes: (i) for large values of the frequency noise variance, the channel behaves similarly to a channel with circularly uniform iid phase noise; (ii) when the frequency noise variance is small, the effect of the additive noise dominates over that of the phase noise, while (iii) for intermediate values of the frequency noise variance, the transmission rate over the phase modulation channel has to be reduced due to the presence of phase noise.

We consider the remote source coding setting in which a source realization is estimated from a lossy compressed sequence of noisy observations. Unlike in the optimal remote source coding problem, however, the encoder is bound to use good codes with respect to the observation sequence, i.e., codes that are optimal for the lossy reconstruction of the observation, rather than the remote source. This encoding strategy is denoted as the compress-and-estimate (CE) scheme. For the case of an i.i.d source observed through a memoryless channel, we show that the distortion in the CE scheme is characterized by a single-letter expression, referred to as the CE distortion-rate function (CE-DRF). In particular, we show that the CE-DRF can be attained by estimating the source from the output of a remote encoder employing any sequence of good codes with respect to the observation sequence. In addition, we show that the limiting distortion in estimating any finite sub-block of the source realization from the output of a remote encoder employing good codes, averaged over all sub-blocks, is also bounded by the CE-DRF.

The discrete-time Wiener phase noise channel with an integrate-and-dump multi-sample receiver is studied. A novel outer bound on the capacity with an average input power constraint is derived as a function of the oversampling factor. This outer bound yields the degrees of freedom for the scenario in which the oversampling factor grows with the transmit power P as Pα. The result shows, perhaps surprisingly, that the largest pre-log that can be attained with phase modulation at high signal-to-noise ratio is at most 1/4.

The capacity of a discrete-time, multi-input multi-output (MIMO) channel with output quantization is investigated for different receiver architectures. A general framework for low-resolution quantization is proposed in which the antenna outputs are processed by analog combiners and sign quantizers are used for analog-to-digital conversion. The configuration of the analog combiners is chosen as a function of the channel realization so that the transmission rate can be maximized over the set of available configurations. To exemplify the proposed approach, four analog receiver architectures are considered: (a) sign quantization of the antenna outputs, (b) single antenna selection, (c) multiple antenna selection, and (d) linear processing of the antenna outputs. In each scenario, capacity is investigated as a function of the transmit power, the number of transmit/receive antennas and sign quantizers. In particular, it is shown that architecture (a) is sufficient to approach the optimal high signal-to-noise ratio (SNR) performance for a MIMO receiver in which the number of receive antennas is larger than the number of sign quantizers. Numerical evaluations of the average performance are presented for the case in which the channel gains are i.i.d. Gaussian distributed.

In this paper, the approximate capacity of the 3-receiver “writing on random dirty paper” (WRDP) channel is derived. In the M-receiver WRDP channel, the channel output is obtained as the sum of the channel input, white Gaussian noise and a channel state sequence randomly selected among a set of M independent Gaussian sequences. The transmitter has non-causal knowledge of the set of possible state sequences but does not know which one is selected to produce the channel output. In the following, we derive upper and lower bounds to the capacity of the 3-receiver WRDP channel which are to within a distance of at most 3 bits-per-channel-use (bpcu) for all channel parameters. In the achievability proof, the channel input is composed of the superposition of three codewords: the receiver opportunistically decodes a different set of codewords, depending on the variance of the channel state appearing in the channel output. Time-sharing among multiple transmission phases is employed to guarantee that transmitted message can be decoded regardless of the state realization. In the converse proof, we derive a novel outer bound which matches the pre-log coefficient arising in the achievability proof due to time-sharing. Although developed for the case of three possible state realizations, our results can be extended the general WRDP.

We consider the remote vector source coding problem in which a vector Gaussian source is estimated from noisy linear measurements. For this problem, we derive the performance of the compress-and-estimate (CE) coding scheme and compare it to the optimal performance. In the CE coding scheme, the remote encoder compresses the noisy source observations so as to minimize a local distortion measure, independent from the joint distribution between the source and the observations. In reconstruction, the decoder, having full knowledge of the joint distribution of the source and observations, estimates the original source realization from the lossy-compressed noisy observations. For the CE scheme in the vector Gaussian case, we show that, if the code rate is less than a specific threshold, then the CE coding scheme attains the same performance as the optimal coding scheme. For code rates above this threshold, we introduce lower and upper bounds on the performance gap between the CE and the optimal scheme. The case of a two-dimensional Gaussian source observed through two noisy measurements is studied to illustrate the behavior of the performance gap.

The “carbon copy onto dirty paper” (CCDP) channel is the compound “writing on dirty paper” channel in which the channel output is obtained as the sum of the channel input, white Gaussian noise and a Gaussian state sequence randomly selected among a set possible realizations. The transmitter has non-causal knowledge of the set of possible state sequences but does not know which sequence is selected to produce the channel output. We study the capacity of the CCDP channel for two scenarios: 1) the state sequences are independent and identically distributed; and 2) the state sequences are scaled versions of the same sequence. In the first scenario, we show that a combination of superposition coding, time-sharing, and Gel'fand-Pinsker binning is sufficient to approach the capacity to within 3 bits per channel use for any number of possible state realizations. In the second scenario, we derive capacity to within 4 bits per channel use for the case of two possible state sequences. This result is extended to the CCDP channel with any number of possible state sequences under certain conditions on the scaling parameters, which we denote as “strong fading” regime. We conclude by providing some remarks on the capacity of the CCDP channel in which the state sequences have any jointly Gaussian distribution.

We consider a multiterminal source coding problem in which a random source signal is estimated from encoded versions of multiple noisy observations. Each encoded version, however, is compressed so as to minimize a local distortion measure, defined only with respect to the distribution of the corresponding noisy observation. The original source is then estimated from these compressed noisy observations. We denote the minimal distortion under this coding scheme as the compress-and-estimate distortion-rate function (CE-DRF). We derive a single-letter expression for the CE-DRF in the case of an i.i.d source. We evaluate this expression for the case of a Gaussian source observed through multiple parallel AWGN channels and quadratic distortion and in the case of a non-uniform binary i.i.d source observed through multiple binary symmetric channels under Hamming distortion. For the case of a Gaussian source, we compare the performance for centralized encoding versus that of distributed encoding. In the centralized encoding scenario, when the code rates are sufficiently small, there is no loss of performance compared to the indirect source coding distortion-rate function, whereas distributed encoding achieves distortion strictly larger then the optimal multiterminal source coding scheme. For the case of a binary source, we show that even with a single observation, the CE-DRF is strictly larger than that of indirect source coding.

Interference pre-cancellation as in the “writing onto dirty paper” channel crucially depends on the transmitter having exact knowledge of the way in which input and channel state combine to produce the channel output. The presence of even a small amount of uncertainty in such knowledge, gravely hampers the ability of the encoder to pre-code its transmissions against the channel state. This is particularly disappointing as it implies that interference pre-coding in practical systems is effective only when the channel estimates have very high precision, a condition which is generally unattainable in wireless environments. In this paper we show that state decoding, instead of state pre-cancellation, can be approximately optimal for a channel with discrete states when only partial channel knowledge is available. More specifically, we consider a variation of the “writing onto dirty paper” channel in which a discrete-valued state sequence is multiplied by a fast fading process and derive conditions on the fading distribution for which state decoding closely approaches capacity. This channel model is a special case of the Gelf'and-Pinsker channel and our results show an instance of this problem in which state decoding is approximately optimal.

We consider a multiterminal source coding problem in which a source is estimated at a central processing unit from lossy-compressed remote observations. Each lossy-encoded observation is produced by a remote sensor. The sensor first obtains a noisy version of the source, then compresses this observation based on minimizing a local distortion measure that depends only on the marginal distribution of its observation. The central node, on the other hand, has knowledge of the joint distribution of the source and all the observations and produces the source estimate that minimizes a different distortion measure between the source and its reconstruction. In this paper, we investigate the problem of optimally choosing the rate of each lossy-compressed remote estimate so as to minimize the distortion at the central processor, subject to bound on the sum of the communication rate between the sensors and the central unit. We focus, in particular, on two models of practical relevance: the case of a Gaussian source observed in additive Gaussian noise and reconstructed under quadratic distortion, and the case of a binary source observed in bit-flipping noise and reconstructed under Hamming distortion. In both scenarios we show that there exist regimes under which having more remote encoders does not reduce the source distortion. In other words, having fewer, high-quality remote estimates provides a smaller distortion than having more, lower-quality estimates.

A unified graphical approach to random coding for any memoryless, single-hop, K -user channel with or without common information is defined through two steps. The first step is user virtualization. Each user is divided into multiple virtual sub-users according to a chosen rate-splitting strategy. This results in an enhanced channel with a possibly larger number of users for which more coding possibilities are available and for which common messages to any subset of users can be encoded. Following user virtualization, the message of each user in the enhanced model is coded using a chosen combination of coded time-sharing, superposition coding, and joint binning. A graph is used to represent the chosen coding strategies. Nodes in the graph represent codewords, while edges represent coding operations. This graph is used to construct a graphical Markov model, which illustrates the statistical dependence among codewords that can be introduced by the superposition coding or joint binning. Using this statistical representation of the overall codebook distribution, the error probability of the code is shown to vanish through a unified analysis. The rate bounds that define the achievable rate region are obtained by linking the error analysis to the properties of the graphical Markov model. This proposed framework makes it possible to numerically obtain an achievable rate region by specifying a user virtualization strategy and describing a set of coding operations. The union of these rate regions defines the maximum achievable rate region of our unified coding strategy. The achievable rates obtained based on this unified graphical approach to random coding encompass the best random coding achievable rates for all memoryless single-hop networks known to date, including broadcast, multiple access, interference, and cognitive radio channels, as well as new results for topologies not previously studied, as we illustrate with several examples.

Costa's “writing on dirty paper” result establishes that full state pre-cancellation can be attained in the Gel'fand-Pinsker problem with additive state and additive white Gaussian noise. This result holds under the assumptions that full channel knowledge is available at both the transmitter and the receiver. In this work we consider the scenario in which the state is multiplied by an ergodic fading process which is not known at the encoder. We study both the case in which the receiver has knowledge of the fading and the case in which it does not: for both models we derive inner and outer bounds to capacity and determine the distance between the two bounds when possible. For the channel without fading knowledge at either the transmitter or the receiver, the gap between inner and outer bounds is finite for a class of fading distributions which includes a number of canonical fading models. In the capacity approaching strategy for this class, the transmitter performs Costa's pre-coding against the mean value of the fading times the state while the receiver treats the remaining signal as noise. For the case in which only the receiver has knowledge of the fading, we determine a finite gap between inner and outer bounds for two classes of discrete fading distribution. The first class of distributions is the one in which there exists a probability mass larger than one half while the second class is the one in which the fading is uniformly distributed over values that are exponentially spaced apart. Unfortunately, the capacity in the case of a continuous fading distribution remains very hard to characterize.

The impact of phase fading and side information on the classic Costa's dirty paper coding channel is studied. A variation of this model is considered in which the channel state is affected by a phase fading sequence which is known at the receiver but not at the transmitter. Although the capacity of this channel has been established, it is expressed as the solution of the maximization which cannot be easily determined. To circumvent such difficulty, we derive alternative inner and outer bounds to capacity and determine a regime in which the two expressions are to within a finite distance. We consider two distributions of the phase fading process: circular binomial and circular uniform. For circular binomial fading we show that binning with Gaussian signaling approaches capacity, as in the channel without phase fading. When fading is circular uniform, instead, binning with Gaussian signaling is no longer effective and novel interference avoidance strategies are developed for this case.

The classic “writing on dirty paper” capacity result establishes that full state pre-cancellation can be attained in Gelfand-Pinsker problem with additive state and additive white Gaussian noise. This result holds under the assumption that both the transmitter and the receiver have perfect knowledge of the channel. We are interested in characterizing capacity under the more realistic assumption that only partial channel knowledge is available at the transmitter. To this end we study the “dirty paper channel with slow fading dirt”, a variation of the dirty paper channel in which the state sequence is multiplied by a slow fading value known only at the receiver. For this model we establish two approximate characterizations of capacity, one for the case in which fading takes only two values and one for the case in which fading takes M possible values but these values are greatly spaced apart. For both results, a naive strategy in which the encoder pre-codes against different fading realizations in different time slots is sufficient to approach capacity.

The capacity of the multiantenna Gaussian cognitive interference channel is studied. The cognitive interference channel is a variation of the classical two-users interference channel in which one of the transmitters, the cognitive transmitter, is also provided with the message of the second transmitter, the primary transmitter. We study the capacity of the multiple-input multiple-output Gaussian model, that is the channel in which the inputs are vectors and the outputs are obtained as linear combinations of the channel inputs plus an additive complex Gaussian noise. This channel models a wireless scenario in which transmitters and receivers have multiple antennas. For this channel, we derive capacity to within an additive gap, that is we show that inner and outer bounds to capacity lie to within a constant distance of each other. The gap between the inner and outer bounds depends on the number of antennas at the cognitive receiver and both bounds can be easily evaluated by considering jointly Gaussian inputs. We also derive capacity to within a constant multiplicative factor of two, that is we show that the ratio between inner and outer bound is at most two. The additive gap well-characterizes the capacity at high SNR, while the multiplicative gap is useful at low SNR. We also derive the exact capacity for a subset of the "strong interference" regime: in this subset, the primary transmitter can decode the cognitive message without loss of optimality. This new capacity result extends and generalizes previously known capacity results, in particular, the capacity in the "very strong interference" and the "primary decodes cognitive" regimes.

The impact of cognitive radio techniques on the energy efficiency of a downlink cellular system in which multiple relays assist the transmission of the base station toward multiple receivers is studied. In particular, the fundamental tradeoff between the power consumption at the base station and the level of cooperation at the relay nodes is investigated. By increasing its transmit power, the base station can distribute the same message to multiple relays. In turn, the common knowledge at the relays enables cooperation, which results in a reduction in the power consumption due to interference management and coherent combining gains. This implies that the overall power efficiency can potentially be improved by an increase in the power consumption at the base station. We employ an information-theoretical analysis of the attainable power efficiency based on the chain graph representation of achievable schemes. This novel theoretical tool uses a graphical Markov model to represent coding operations and allows for the automatic derivation of achievable rate regions for general networks. This approach provides an effective tool to analyze the relationship between the energy consumption at the base station and power savings provided by relay cooperation through the use of transmission strategies such as superposition coding, interference decoding and rate-splitting. We present numerical evaluations for the scenario in which two relay nodes aid the communication between the base station and three receivers. These evaluations show that cooperative strategies at the relays provide clear advantages as compared to the non-cooperative scenario for varying channel conditions and target rates.

The interference channel with a cognitive relay (IFC-CR) consists of the classical IFC with two independent source-destination pairs whose communication are aided by an additional node, referred to as the CR, that has a priori knowledge of both sources' messages. This a priori message knowledge is termed cognition and idealizes the relay learning the messages of the two sources from their transmissions over a wireless channel. This paper presents improved outer and inner bounds on the capacity region of the general memoryless IFC-CR that are shown to be tight for certain classes of channels. The new outer bound follows from arguments originally devised for broadcast channels, among which Sato's observation that the capacity region of channels with noncooperative receivers only depends on conditional marginal distributions of the channel output, not on their conditional joint distribution. A simplified expression for the inner bound is derived, which contains all previously proposed coding schemes. The new inner and outer bounds coincide for a class of channels satisfying some strong interference condition, i.e., for these channels there is no loss in optimality if both destinations decode both messages. This result parallels analogous results for the classical interference channel and for the cognitive interference channel and is the first known capacity result for the general IFC-CR. Numerical evaluations of the proposed inner and outer bounds are presented for the additive white Gaussian noise case.

This paper considers a variation of the classical two-user interference channel where the communication of two interfering source-destination pairs is aided by an additional node that has a priori knowledge of the messages to be transmitted, which is referred to as the cognitive relay. For this interference channel with a cognitive relay (ICCR), novel outer bounds and capacity region characterizations are derived. In particular, for the class of injective semi-deterministic ICCRs, a sum-rate upper bound is derived for the general memoryless ICCR and further tightened for the linear deterministic approximation (LDA) of the Gaussian noise channel at high SNR, which disregards the noise and focuses on the interaction among the users' signals. The capacity region of the symmetric LDA is completely characterized except for the regime of moderately weak interference and weak links from the CR to the destinations. The insights gained from the analysis of the LDA are then translated back to the symmetric Gaussian noise channel (GICCR). For the symmetric GICCR, an approximate characterization (to within a constant gap) of the capacity region is provided for a parameter regime where capacity was previously unknown. The approximately optimal scheme suggests that message cognition at a relay is beneficial for interference management as it enables simultaneous over the air neutralization of the interference at both destinations.

We study transmit power minimization in the two-user Z Interference Channel (ZIC). When the interference link gain is strong, the capacity of the ZIC has been fully characterized. For this strong interference regime, we derive the closed-form solution of the minimum required transmit power to achieve a given rate pair, and show that the resulting power allocation between the two users is not necessarily unique. When the interference link gain is weak, the capacity of the ZIC is still an open problem to date. For this weak interference regime, we develop an inner and outer bound for the required transmit power to achieve a given rate pair, and characterize the constant power ratio relation between the two bounds. In contrast to the strong interference case, rate splitting is necessary for power minimization in this regime. The optimal power-minimizing rate splitting solution is then derived, and performance in terms of total transmit power required for a given rate pair is analyzed.

In this paper a new similarity index for neurograms is proposed. This index is inspired by the Needleman-Wunsch algorithm which determines the minimum number of operations to transform a vector into another in terms of insertions, deletions and substitutions. The Needleman-Wunsch algorithm can be extended to the two dimensional case and the number of transformations required to change a matrix into another is used to define a measure of similarity. This similarity measure is applied to neurograms and optimized to perform prediction of speech intelligibility in noise. Word recognition scores for for speech samples in noise are evaluated using the proposed similarity index, showing a clear improvement in speech intelligibility estimation with respect to other neurogram similarity metrics in the literature. The proposed similarity index is not restricted to a certain time resolution and could serve to evaluate neurogram similarity with respect to temporal fine structure in future.

A downlink cellular system in which multiple relays assist the transmission of the base station is considered. Cooperation strategies based on superposition coding are derived for this network. Superposition coding is attained by sending each message to one or more relays while satisfying the rate constraints of the base station-to-relay links. The chain graph representation of achievable schemes is used to maximize the network throughput over the set of feasible transmission strategies based on superposition coding. Rate advantages as compared to the non-cooperative scenario are obtained under varying relay positions and available power at the base station and relay nodes.

The cognitive interference channel is a variation of the classical interference channel in which one of the transmitters, the cognitive transmitter, has full and a priori knowledge of the message of the other user, the primary user. This additional knowledge is termed cognition and idealizes the cognitive transmitter learning the messages of the primary user by overhearing its transmissions over a wireless channel. This paper studies the multiple-input multiple-output cognitive interference channel and derives inner and outer bounds for the capacity of this channel model as well as approximate characterizations of the capacity region. In particular, it is shown that capacity can be achieved to within an additive gap which depends on the number of antennas at the cognitive decoder and to within a constant multiplicative factor of two.

The capacity region of the interference channel with common messages is studied. This channel model is a variation of the classical two user interference channel modified such that each encoder has both a private message as well as a common message to be decoded at both receivers. Achievable rates for this channel model can be characterized by the classical Han-Kobayashi achievable rate region for the interference channel in which, at each encoder, the codeword embedding the private message is superimposed over the codeword for the common message. We show that the achievable rates for this region can be improved upon by rate-sharing, which consist of transmitting part of the private message into the common codeword. This improved region is shown to approach the capacity for a class of injective semi-deterministic channels. Moreover, we show that the Fourier Motzkin elimination of the Han-Kobayashi with rate-sharing contains less rate bounds than the one without rate-sharing. This approach provides an alternative proof of the simplification of the Han-Kobayashi originally shown by Chong et al. This result is particularly interesting as it shows that simplifications in the spirit of Chong et al. for general channels can be performed through rate-sharing and Fourier-Motzkin elimination. This approach can be easily implemented algorithmically and is relevant in the context of the automatic derivation of achievable rate regions.

In this correspondence information theoretical tools are used to investigate the statistical properties of modeled cochlear nucleus globular bushy cell spike trains. The firing patterns are obtained from a simulation software that generates sample spike trains from any auditory input. Here we analyze for the first time the responses of globular bushy cells to voiced and unvoiced speech sounds. Classical entropy estimates, such as the direct method, are improved upon by considering a time-varying and time-dependent entropy estimate. With this method we investigated the relationship between the predictability of the neuronal response and the frequency content in the auditory signals. The analysis quantifies the temporal precision of the neuronal coding and the memory in the neuronal response.

In this paper we study the cognitive interference channel with a common message, a variation of the classical cognitive interference channel in which the cognitive message is decoded at both receivers. We derive the capacity for the semi-deterministic model, a class of channels in which the output at the cognitive decoder is a deterministic function of the channel inputs. We also show capacity to within a constant gap and a constant factor for the Gaussian channel. Most of these results are shown using an interesting transmission scheme in which the cognitive message, decoded at both receivers, is also pre-coded against the interference experienced at the cognitive receiver. The pre-coding of the cognitive message does not allow the primary decoder to reconstruct the interfering signal; the cognitive message acts instead as a side information at the primary receiver when decoding its intended message.

The chain graph representations of an achievable scheme is a recently introduced theoretical tool to derive achievable regions based on superposition coding and binning for a general, single-hop, multi-terminal network. It allows for a compact representation of complex transmission strategies and the derivation of the corresponding achievable region for a large class of channels. In this paper we extend the original concept to include a new random coding technique that generalizes superposition coding and binning. With this coding strategy, one generates a top codebook conditionally dependent on the bottom codeword and successively uses binning to impose a different conditional distribution between top and bottom codewords. The region achieved with this strategy relates to the Kullback-Leibler divergence between the distribution of the codewords at generation and the distribution after binning.

The cognitive interference channel models cognitive overlay radio systems, in which cognitive radios overhear the transmission of neighboring nodes. For the Gaussian case capacity is known in three subsets of the parameter space: the “weak interference”, “very strong interference” and “primary decodes cognitive” regime. Capacity in the “very strong interference” regime is achieved by superposing the cognitive message over the primary message while in the “primary decodes cognitive” regime the cognitive message is binned against the primary message. This paper provides a new capacity result obtained by combining the capacity achieving schemes in these two regimes thus generalizing and extending these results. Interestingly, the capacity achieving strategy for a given channel also depends on the level of cooperation among the users: that is, either superposition coding or binning is employed depending on the amount of power allotted by the cognitive transmitter to aid the primary user.

The capacity of the Gaussian cognitive interference channel, a variation of the classical two-user interference channel where one of the transmitters (referred to as cognitive) has knowledge of both messages, is known in several parameter regimes but remains unknown in general. This paper provides a comparative overview of this channel model as it proceeds through the following contributions. First, several outer bounds are presented: (a) a new outer bound based on the idea of a broadcast channel with degraded message sets, and (b) an outer bound obtained by transforming the channel into channels with known capacity. Next, a compact Fourier-Motzkin eliminated version of the largest known inner bound derived for the discrete memoryless cognitive interference channel is presented and specialized to the Gaussian noise case, where several simplified schemes with jointly Gaussian input are evaluated in closed form and later used to prove a number of results. These include a new set of capacity results for: (a) the “primary decodes cognitive” regime, a subset of the “strong interference” regime that is not included in the “very strong interference” regime for which capacity was known, and (b) the “S-channel in strong interference” in which the primary transmitter does not interfere with the cognitive receiver and the primary receiver experiences strong interference. Next, for a general Gaussian channel the capacity is determined to within one bit/s/Hz and to within a factor two regardless of the channel parameters, thus establishing rate performance guarantees at high and low SNR, respectively. The paper concludes with numerical evaluations and comparisons of the various simplified achievable rate regions and outer bounds in parameter regimes where capacity is unknown, leading to further insight on the capacity region.

The cognitive interference channel (C-IFC) consists of a classical two-user interference channel in which the message of one user (the "primary" user) is non-causally available at the transmitter of the other user (the "cognitive" user). We obtain the capacity of the semi-deterministic C-IFC: a discrete memoryless C-IFC in which the cognitive receiver output is a noise-less deterministic function of the channel inputs. We then use the insights obtained from the capacity-achieving scheme for the semi-deterministic model to derive new, unified and tighter constant gap results for the complex-valued Gaussian C-IFC. We prove: (1) a constant additive gap (difference between inner and outer bounds) of half a bit/sec/Hz per real dimension, of relevance at high SNRs, and (b) a constant multiplicative gap (ratio between outer and inner bounds) of a factor two, of relevance at low SNRs.

This work proposes a novel outer bound for the Gaussian cognitive interference channel in strong interference at the primary receiver based on the capacity of a multi-antenna broadcast channel with degraded message set. It then shows that for the Z-channel, i.e., when the secondary receiver experiences no interference and the primary receiver experiences strong interference, the proposed outer bound not only is the tightest among known bounds but is actually achievable for sufficiently strong interference. The latter is a novel capacity result that from numerical evaluations appears to be generalizable to a larger (i.e., non-Z) class of Gaussian channels.

The InterFerence Channel with a Cognitive Relay (IFC-CR) consists of a classical two-user interference channel in which the two independent messages are also non-causally known at a cognitive relay node. In this work a special class of IFC-CRs in which the sources do not create interference at the non-intended destinations is analyzed. This special model results in a channel with two non-interfering point-to-point channels whose transmission is aided by an in-band cognitive relay, which is thus referred to as the Parallel Channel with a Cognitive Relay (PC-CR). We determine the capacity of the PC-CR channel to within 3 bits/s/Hz for all channel parameters. In particular, we present several new outer bounds which we achieve to within a constant gap by proper selection of Gaussian input distributions in a simple rate-splitting and superposition coding-based inner bound. The inner and outer bounds are numerically evaluated to show that the actual gap can be far less than 3 bits/s/Hz.

The interference channel with a cognitive relay consists of a classical interference channel with two source-destination pairs and with an additional cognitive relay that has a priori knowledge of the sources' messages and aids in the sources' transmission. We derive a new outer bound for this channel using an argument originally devised for the “more capable” broadcast channel, and show the achievability of the proposed outer bound for a class of channels where there is no loss in optimality if both destinations decode both messages. This result is analogous to the “very strong interference” capacity result for the classical interference channel and for the cognitive interference channel, and is the first capacity known capacity result for the general interference channel with a cognitive relay.

The cognitive interference channel is a two-user interference channel in which one transmitter is non-causally provided with the message of the other transmitter. This channel model has been extensively studied in the past years and capacity results have been proved for certain classes of channels. This paper presents new inner and outer bounds for the capacity region of the cognitive interference channel, as well as new capacity results. Previously proposed outer bounds are expressed in terms of auxiliary random variables for which no cardinality constraint of their alphabet is known. Consequently, it is not possible to evaluate such outer bounds explicitly for a given channel. The outer bound derived in this work is based on an idea originally devised by Sato for channels without receiver cooperation and results in an outer bound that does not contain auxiliary random variables, thus allowing it to be more easily evaluated. The inner bound presented in this work-which includes rate splitting, superposition coding, a broadcast channel-like binning scheme and Gel'fand Pinsker coding-is the largest known to date and is explicitly shown to include all previously proposed achievable rate regions. The novel inner and outer bounds are shown to coincide in certain cases. In particular, capacity is proved for a class of channels in the so-called “better cognitive decoding” regime, which includes the regimes in which capacity was known. Finally, the capacity region of the semi-deterministic cognitive interference channel, in which the signal at the cognitive receiver is an arbitrary deterministic function of the channel inputs, is established.

In this paper, we first present an outer bound for a general interference channel with a cognitive relay, i.e., a relay that has non-causal knowledge of both independent messages transmitted in the interference channel. This outer bound reduces to the capacity region of the deterministic broadcast channel and of the deterministic cognitive interference channel the through nulling of certain channel inputs. It does not, however, reduce to that of certain deterministic interference channels for which capacity is known. As such, we subsequently tighten the bound for channels whose outputs satisfy an “invertibility” condition. This second outer bound now reduces to the capacity of the special class of deterministic interference channels for which capacity is known. The second outer bound is further tightened for the high-SNR deterministic approximation of the Gaussian channel by exploiting the special structure of the interference. We provide an example that suggests that this third bound is tight in at least some parameter regimes for the high-SNR deterministic approximation of the Gaussian channel. Another example shows that the third bound is capacity in the special case where there are no direct links between the non-cognitive transmitters.