We study a one-shot joint source-channel coding setting where the source is encoded once and broadcast to $K$ decoders through independent channels. Success is predicated on at least one decoder recovering the source within a maximum distortion constraint. We find that in the one-shot regime, utilizing disjoint codebooks at each decoder yields a codebook diversity gain, distinct from the channel diversity gain that may be expected when several decoders observe independent realizations of the channel's output but share the same codebook. Coding schemes are introduced that leverage this phenomenon, where first- and second-order achievability bounds are derived via an adaptation of the Poisson matching lemma which allows for multiple decoders using disjoint codebooks. We further propose a hybrid coding scheme that partitions decoders into groups to optimally balance codebook and channel diversity. Numerical results on the binary symmetric channel demonstrate that the hybrid approach outperforms strategies where the decoders' codebooks are either fully shared or disjoint.

We consider multi-draft speculative sampling, where the proposal sequences are sampled independently from different draft models. At each step, a token-level draft selection scheme takes a list of valid tokens as input and produces an output token whose distribution matches that of the target model. Previous works have demonstrated that the optimal scheme (which maximizes the probability of accepting one of the input tokens) can be cast as a solution to a linear program. In this work we show that the optimal scheme can be decomposed into a two-step solution: in the first step an importance sampling (IS) type scheme is used to select one intermediate token; in the second step (single-draft) speculative sampling is applied to generate the output token. For the case of two identical draft models we further 1) establish a necessary and sufficient condition on the distributions of the target and draft models for the acceptance probability to equal one and 2) provide an explicit expression for the optimal acceptance probability. Our theoretical analysis also motives a new class of token-level selection schemes based on weighted importance sampling. Our experimental results demonstrate consistent improvements in the achievable block efficiency and token rates over baseline schemes in a number of scenarios.

The Importance Matching Lemma (IML) [1] is a recently introduced importance sampling-based technique for distributed compression, offering a finite-proposals alternative to the Poisson Matching Lemma (PML). Unlike PML, which relies on an infinite proposals formulation and can encounter termination issues, IML operates directly on finite samples, ensuring practical feasibility in real-world scenarios. However, its current formulation is restricted to exact matching and does not extend to list decoding, a capability available under PML. In this work, we extend IML to support list decoding by leveraging results from the order statistics of exponential random variables with different rates. We provide a detailed theoretical analysis of the proposed extension and derive list-decoding guarantees in the context of channel coding. Our results demonstrate a novel application of importance sampling and list-decoding techniques, expanding the scope of IML to a wider range of coding scenarios.

Parameter-Efficient Fine-Tuning (PEFT) methods like Low-Rank Adaptation (LoRA) optimize federated training by reducing computational and communication costs. We propose RoLoRA, a federated framework using alternating optimization to fine-tune LoRA adapters. Our approach emphasizes the importance of learning up and down projection matrices to enhance expressiveness and robustness. We use both theoretical analysis and extensive experiments to demonstrate the advantages of RoLoRA over prior approaches that either generate imperfect model updates or limit expressiveness of the model. We provide a theoretical analysis on a linear model to highlight the importance of learning both the down-projection and up-projection matrices in LoRA. We validate the insights on a non-linear model and separately provide a convergence proof under general conditions. To bridge theory and practice, we conducted extensive experimental evaluations on language models including RoBERTa-Large, Llama-2-7B on diverse tasks and FL settings to demonstrate the advantages of RoLoRA over other methods.

We study channel simulation and distributed matching, two fundamental problems with several applications to machine learning, using a recently introduced generalization of the standard rejection sampling (RS) algorithm known as Ensemble Rejection Sampling (ERS). For channel simulation, we propose a new coding scheme based on ERS that achieves a near-optimal coding rate. In this process, we demonstrate that standard RS can also achieve a near-optimal coding rate and generalize the result of Braverman and Garg (2014) to the continuous alphabet setting. Next, as our main contribution, we present a distributed matching lemma for ERS, which serves as the rejection sampling counterpart to the Poisson Matching Lemma (PML) introduced by Li and Anantharam (2021). Our result also generalizes a recent work on importance matching lemma (Phan et al, 2024) and, to our knowledge, is the first result on distributed matching in the family of rejection sampling schemes where the matching probability is close to PML. We demonstrate the practical significance of our approach over prior works by applying it to distributed compression. The effectiveness of our proposed scheme is validated through experiments involving synthetic Gaussian sources and distributed image compression using the MNIST dataset.

We study a relaxation of the problem of coupling probability distributions — a list of samples is generated from one distribution and an *accept* is declared if any one of these samples is identical to the sample generated from the other distribution. We propose a novel method for generating samples, which extends the Gumbel-max sampling suggested in Daliri et al. (2025) for coupling probability distributions. We also establish a corresponding lower bound on the acceptance probability, which we call the *list matching lemma*. We next discuss two applications of our setup. First, we develop a new mechanism for multi-draft speculative sampling that is simple to implement and achieves performance competitive with baselines such as SpecTr and SpecInfer across a range of language tasks. Our method also guarantees a certain degree of *drafter invariance* with respect to the output tokens which is not supported by existing schemes. We also provide a theoretical lower bound on the token level acceptance probability. As our second application, we consider distributed lossy compression with side information in a setting where a source sample is compressed and available to multiple decoders, each with independent side information. We propose a compression technique that is based on our generalization of Gumbel-max sampling and show that it provides significant gains in experiments involving synthetic Gaussian sources and the MNIST image dataset.

This survey article focuses on the emerging connections between machine learning and data compression. While the fundamental limits of classical (lossy) data compression are well-established through rate-distortion theory, recent advancements have uncovered new theoretical analyses and application areas inspired by machine learning. We review recent works on task-based and goal-oriented compression, rate-distortion-perception theory, and compression for estimation and inference. Deep learning-based approaches have provided natural, data-driven methods for compression. Accordingly, we survey recent efforts in applying deep learning techniques to task-based or goal-oriented compression, as well as image/video compression and transmission. Additionally, we discuss the potential use of large language models for text compression. Finally, we outline future research directions in this promising field.

In the context of lossy compression, Blau & Michaeli [1] adopt a mathematical notion of perceptual quality and define the information rate-distortion-perception function, generalizing the classical rate-distortion tradeoff. We consider the notion of universal representations in which one may fix a rate and an encoder then vary the decoder to achieve any point within a collection of distortion and perception constraints. We prove that the corresponding information-theoretic universal rate-distortion-perception function is operationally achievable in an approximate sense. Under MSE distortion, we show that the entire distortion-perception tradeoff of a Gaussian source can be achieved by a single encoder of the same rate asymptotically. We then characterize the achievable distortion-perception region for a fixed representation in the case of arbitrary distributions, and identify conditions under which the aforementioned results continue to hold approximately. Finally, we extend our notion of universality to the case where the rate is no longer fixed and additional bits can be sent at a second stage, generalizing the classical theory of successive refinement [2] with perception constraints. This motivates the study of practical constructions that are approximately universal across the RDP tradeoff, thereby alleviating the need to design a new encoder for each objective. We provide experimental results on MNIST and SVHN suggesting that on image compression tasks, the operational tradeoffs achieved by machine learning models with a fixed encoder suffer only a small penalty when compared to their variable encoder counterparts.

We propose two extensions to existing importance sampling based methods for lossy compression. First, we introduce an importance sampling based compression scheme that is a variant of ordered random coding (Theis and Ahmed, 2022) and is amenable to direct evaluation of the achievable compression rate for a finite number of samples. Our second and major contribution is the \emph{importance matching lemma}, which is a finite proposal counterpart of the recently introduced {Poisson matching lemma} (Li and Anantharam, 2021). By integrating with deep learning, we provide a new coding scheme for distributed lossy compression with side information at the decoder. We demonstrate the effectiveness of the proposed scheme through experiments involving synthetic Gaussian sources, distributed image compression with MNIST and vertical federated learning with CIFAR-10.

We propose a secure inference protocol for a distributed setting involving a single server node and multiple client nodes. We assume that the observed data vector is partitioned across multiple client nodes while the deep learning model is located at the server node. Each client node is required to encrypt its portion of the data vector and transmit the resulting ciphertext to the server node. The server node is required to collect the ciphertexts and perform inference in the encrypted domain. We demonstrate an application of multi-party homomorphic encryption (MPHE) to satisfy these requirements. We propose a packing scheme, that enables the server to form the ciphertext of the complete data by aggregating the ciphertext of data subsets encrypted using MPHE. While our proposed protocol builds upon prior horizontal federated training protocol [1], we focus on the inference for vertically partitioned data and avoid the transmission of (encrypted) model weights from the server node to the client nodes.

This paper investigates adaptive streaming codes over a three-node relayed network. In this setting, a source transmits a sequence of message packets through a relay under a delay constraint of$T$time slots per packet. The source-to-relay and relay-to-destination links are unreliable and introduce a maximum of$N_{1}$and$N_{2}$packet erasures respectively. Recent work has proposed adaptive (time variant) and nonadaptive (time invariant) code constructions for this setting and has shown that adaptive codes can achieve higher rates. However, the adaptive construction deals with many possibilities, leading to an impractical code with very large block lengths. We therefore propose a simplified adaptive code construction which greatly improves the practicality of the code, with only a small cost to the achievable rates. We analyze our construction in terms of the achievable rates and field size requirements, and perform numerical simulations to estimate packet loss probabilities over statistical channels.

The Poisson Matching Lemma (PML) introduced by Li & Anantharam (IT-Trans 2021) is a powerful technique for one-shot analysis of a variety of multi-terminal source and channel coding problems. In this work we make use of PML to derive one-shot achievability results for unequal message protection with a fixed number of message classes. Our analysis involves revisiting the proof of the PML to account for the error associated with each codebook at the decoder. Our approach leads to compact bounds on the error probability for each message class for arbitrary input distributions and channels. For the example of binary erasure channel, we compare our bounds numerically with prior work [1] and demonstrate improvements in the achievable rate.

This paper studies the rate-distortion-perception (RDP) tradeoff for a memoryless source model in the asymptotic limit of large block-lengths. The perception measure is based on a divergence between the distributions of the source and reconstruction sequences conditioned on the encoder output, first proposed by Mentzer et al. We consider the case when there is no shared randomness between the encoder and the decoder. For the case of discrete memoryless sources we derive a single-letter characterization of the RDP function, in contrast to the marginal-distribution metric case (introduced by Blau and Michaeli), whose RDP characterization remains open when there is no shared randomness. The achievability scheme is based on lossy source coding with a posterior reference map. For the case of continuous valued sources under squared error distortion measure and squared quadratic Wasserstein perception measure we also derive a single-letter characterization and show that a noise-adding mechanism at the decoder suffices to achieve the optimal representation. Interestingly, the RDP function characterized for the case of zero perception loss coincides with that of the marginal metric and further zero perception loss can be achieved with a 3-dB penalty in minimum distortion. Finally we specialize to the case of Gaussian sources, and derive the RDP function for Gaussian vector case and propose a waterfilling like solution. We also partially characterize the RDP function for a mixture of Gaussian vector sources.

We present an optimal method for encoding cluster assignments of arbitrary data sets. Our method, Random Cycle Coding (RCC), encodes data sequentially and sends assignment information as cycles of the permutation defined by the order of encoded elements. RCC does not require any training and its worst-case complexity scales quasi-linearly with the size of the largest cluster. We characterize the achievable bit rates as a function of cluster sizes and number of elements, showing RCC consistently outperforms previous methods while requiring less compute and memory resources. Experiments show RCC can save up to $2$ bytes per element when applied to vector databases, and removes the need for assigning integer ids to identify vectors, translating to savings of up to $70\%$ in vector database systems for similarity search applications.

This paper investigates a novel lossy compression framework operating under logarithmic loss, designed to handle situations where the reconstruction distribution diverges from the source distribution. This framework is especially relevant for applications that require joint compression and retrieval, and in scenarios involving distributional shifts due to processing. We show that the proposed formulation extends the classical minimum entropy coupling framework by integrating a bottleneck, allowing for controlled variability in the degree of stochasticity in the coupling. We explore the decomposition of the Minimum Entropy Coupling with Bottleneck (MEC-B) into two distinct optimization problems: Entropy-Bounded Information Maximization (EBIM) for the encoder, and Minimum Entropy Coupling (MEC) for the decoder. Through extensive analysis, we provide a greedy algorithm for EBIM with guaranteed performance, and characterize the optimal solution near functional mappings, yielding significant theoretical insights into the structural complexity of this problem. Furthermore, we illustrated the practical application of MEC-B through experiments in Markov Coding Games (MCGs) under rate limits. These games simulate a communication scenario within a Markov Decision Process, where an agent must transmit a compressed message from a sender to a receiver through its actions. Our experiments highlighted the trade-offs between MDP rewards and receiver accuracy across various compression rates, showcasing the efficacy of our method compared to conventional compression baseline.

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that there exists a convex loss function for which the stability gap for multiple epochs of SGD with standard heavy-ball momentum (SGDM) becomes unbounded. Then, for smooth Lipschitz loss functions, we analyze a modified momentum-based update rule, i.e., SGD with early momentum (SGDEM) under a broad range of step-sizes, and show that it can train machine learning models for multiple epochs with a guarantee for generalization. Finally, for the special case of strongly convex loss functions, we find a range of momentum such that multiple epochs of standard SGDM, as a special form of SGDEM, also generalizes. Extending our results on generalization, we also develop an upper bound on the expected true risk, in terms of the number of training steps, sample size, and momentum. Our experimental evaluations verify the consistency between the numerical results and our theoretical bounds. SGDEM improves the generalization error of SGDM when training ResNet-18 on ImageNet in practical distributed settings.

This paper studies the rate-distortion-perception (RDP) tradeoff for a memoryless source model in the asymptotic limit of large block-lengths. The perception measure is based on a divergence between the distributions of the source and reconstruction sequences conditioned on the encoder output, first proposed by Mentzer et al. We consider the case when there is no shared randomness between the encoder and the decoder and derive a single-letter characterization of the RDP function, for the case of discrete memoryless sources. This is in contrast to the marginal-distribution metric case (introduced by Blau and Michaeli), whose RDP characterization remains open when there is no shared randomness. The achievability scheme is based on lossy source coding with a posterior reference map. For the case of continuous valued sources under the squared error distortion measure and the squared quadratic Wasserstein perception measure, we also derive a single-letter characterization and show that the decoder can be restricted to a noise-adding mechanism. Interestingly, the RDP function characterized for the case of zero perception loss coincides with that of the marginal metric, and further zero perception loss can be achieved with a 3-dB penalty in minimum distortion. Finally we specialize to the case of Gaussian sources, and derive the RDP function for Gaussian vector case and propose a reverse water-filling type solution. We also partially characterize the RDP function for a mixture of Gaussian vector sources.

Multi-subject fMRI studies are challenging due to the high variability of both brain anatomy and functional brain topographies across participants. An effective way of aggregating multi-subject fMRI data is to extract a shared representation that filters out unwanted variability among subjects. Some recent work has implemented probabilistic models to extract a shared representation in task fMRI. In the present work, we improve upon these models by incorporating temporal information in the common latent structures. We introduce a new model, Shared Gaussian Process Factor Analysis (S-GPFA), that discovers shared latent trajectories and subject-specific functional topographies, while modeling temporal correlation in fMRI data. We demonstrate the efficacy of our model using the time-segment matching experiment on the publicly available Raider dataset. We further test the utility of our model by analyzing its learned model parameters in the large multi-site SPINS dataset, on a social cognition task from participants with and without schizophrenia.

We present a one-shot method for compressing large labeled graphs called Random Edge Coding. When paired with a parameter-free model based on Pólya's Urn, the worst-case computational and memory complexities scale quasi-linearly and linearly with the number of observed edges, making it efficient on sparse graphs, and requires only integer arithmetic. Key to our method is bits-back coding, which is used to sample edges and vertices without replacement from the edge-list in a way that preserves the structure of the graph. Optimality is proven under a class of random graph models that are invariant to permutations of the edges and of vertices within an edge. Experiments indicate Random Edge Coding can achieve competitive compression performance on real-world network datasets and scales to graphs with millions of nodes and edges.

Vertical federated learning (VFL) enables the collaborative training of machine learning (ML) models in settings where the data is distributed amongst multiple parties who wish to protect the privacy of their individual data. Notably, in VFL, the labels are available to a single party and the complete feature set is formed only when data from all parties is combined. Recently, Xu et al. [1] proposed a new framework called FedV for secure gradient computation for VFL using multi-input functional encryption. In this work, we explain how some of the information leakage in Xu et al. can be avoided by using Quadratic functional encryption when training generalized linear models for vertical federated learning.

This paper studies streaming erasure codes in a relayed multicast setting, where a source wishes to transmit a sequence of messages to two different destinations through a common relay. Our construction extends previously proposed works on the single-destination setting studied in Fong et al. and Facenda et al. to the multicast setting, where each destination can recover the source packets with a correspondingly different delay. A key property of our construction is that it does not require prior knowledge of the maximum number of erasures on the relay-destination link. Instead, it enables the recovery of the source stream with a decoding delay that depends on the number of erasures on the relay-destination links. We demonstrate that if some divisibility conditions are satisfied, then the proposed construction can simultaneously achieve the single-destination delay in Fong et al. for both receivers. Finally we also explain how our proposed construction can be applied in the setting of a single destination when the maximum number of erasures on the relay-destination link is not known beforehand.

We study causal, low-latency, sequential video compression when the output is subjected to both a mean squared-error (MSE) distortion loss as well as a perception loss to target realism. Motivated by prior approaches, we consider two different perception loss functions (PLFs). The first, PLF-JD, considers the joint distribution (JD) of all the video frames up to the current one, while the second metric, PLF-FMD, considers the framewise marginal distributions (FMD) between the source and reconstruction. Using information theoretic analysis and deep-learning based experiments, we demonstrate that the choice of PLF can have a significant effect on the reconstruction, especially at low-bit rates. In particular, while the reconstruction based on PLF-JD can better preserve the temporal correlation across frames, it also imposes a significant penalty in distortion compared to PLF-FMD and further makes it more difficult to recover from errors made in the earlier output frames. Although the choice of PLF decisively affects reconstruction quality, we also demonstrate that it may not be essential to commit to a particular PLF during encoding and the choice of PLF can be delegated to the decoder. In particular, encoded representations generated by training a system to minimize the MSE (without requiring either PLF) can be {\em near universal} and can generate close to optimal reconstructions for either choice of PLF at the decoder. We validate our results using (one-shot) information-theoretic analysis, detailed study of the rate-distortion-perception tradeoff of the Gauss-Markov source model as well as deep-learning based experiments on moving MNIST and KTH datasets.

This paper studies packet repetition strategies over erasure channels with memory and a long feedback delay. The problem is initially formulated as a communications problem where a source wishes to transmit one message packet to a destination while minimizing both the delay and the number of transmissions. At each time instant, the sender is provided a delayed acknowledgement feedback about past attempts, and must decide whether to attempt a new transmission or not. This problem is then re-formulated as an episodic reinforcement learning problem, where an agent attempts to learn the optimal transmission policy, provided delayed feedback about past transmission attempts. The agent is helped by a channel estimator, which attempts to capture the channel memory and use that to predict probabilities of erasures in a future window. This channel estimator is also data-driven and learns the channel model without anya priorichannel knowledge. The paper presents a lower bound on the achievable trade-off between delay and number of transmissions for any channel modeled as a Markov process. Experimental results show that the combination of the proposed channel estimator and the agent can noticeably outperform naive strategies for channels with memory, and achieves results close to the lower bound.

Many emerging multimedia streaming applications involve multiple users communicating under strict latency constraints. In this paper we study streaming codes for a network involving two source nodes, one relay node and a destination node. In this paper’s setting, each source node transmits a stream of messages, through the relay, to a destination, who is required to decode the messages under a strict delay constraint. For the case of a single source node, a class of streaming codes has been proposed by Fong et al., using the concept of delay-spectrum. The current paper presents a novel framework, which constructs streaming codes for a relayed multi-user setting by sequentially constructing the codes for each link. This requires a characterization of the set of all achievable delay spectra for a given rate, blocklength and number of erasures, beyond the specific choice considered by Fong et al. This characterization is presented in the paper for systematic codes. Using this novel framework, the first proposed scheme involves greedily selecting the rate on the link from relay to destination and using properties of the delay-spectrum to find feasible streaming codes that satisfy the required delay constraints. A closed form expression for the achievable rate region is provided, and conditions for when the proposed scheme is optimal are established by a natural outer bound. The second proposed scheme builds upon this approach, but uses a numerical optimization-based approach to improve the achievable rate region over the first scheme. Experimental results show that the proposed schemes achieve significant improvements over baseline schemes based on single-user codes.

This paper investigates adaptive streaming codes over a three-node relayed network. In this setting, a source node transmits a sequence of message packets to a destination with help of a relay. The source-to-relay and relay-to-destination links are unreliable and introduce at most$N_{1}$and$N_{2}$packet erasures, respectively. The destination node must recover each message packet within a strict delay constraint$T$. The paper presents a new construction of streaming codes for all feasible parameters$\{N_{1}, N_{2}, T\}$. Our work improves upon the construction in Fong et al. by adapting the relaying strategy based on the erasure patterns from source to relay. Specifically, the code employs the notion of symbol estimates, which allows the relay to forward information about symbols before it can decode that symbol, and variable-rate encoding, which decreases the rate used to encode a packet as more erasures affect that packet. The codes proposed in this paper achieve rates higher than the ones proposed by Fong et al. whenever$N_{2} > N_{1}$, and achieve the same rate when$N_{2} \leq N_{1}$, in which case the rate is optimal. The paper also presents an upper bound on the achievable rate that takes into account erasures in both links in order to bound the rate in the second link. The upper bound is shown to be tighter than a trivial bound that considers only the erasures in the second link.

Current methods which compress multisets at an optimal rate have computational complexity that scales linearly with alphabet size, making them too slow to be practical in many real-world settings. We show how to convert a compression algorithm for sequences into one for multisets, in exchange for an additional complexity term that is quasi-linear in sequence length. This allows us to compress multisets of independent and identically distributed symbols at an optimal rate, with computational complexity decoupled from the alphabet size. The key insight is to avoid encoding the multiset directly, and instead compress a proxy sequence, using a technique called ‘bits-back coding’. We demonstrate the method experimentally on two tasks which are intractible with previous optimal-rate methods: compression of multisets of images and JavaScript Object Notation (JSON) files. Code for our experiments is available at https://github.com/facebookresearch/multiset-compression.

This paper proposes a data-driven architecture for zero-delay lossy source coding with side information (i.e., Wyner-Ziv coding) for sources with memory. The overall architecture involves designing suitable filters at the encoder and the decoder and performing fixed-rate scalar quantization followed by one-dimensional binning of quantization indices. Unlike previous work, which uses an exhaustive search to optimize the system parameters, this paper proposes a lower-complexity data-driven method that does not require a priori knowledge of source and side information statistics. The main ingredients of the proposed approach include modeling the quantization process by an additive quantization noise process, modeling the modulo operation by a continuous approximation, and approximating the decoding process by a softmin function, which makes the system amenable to training using stochastic gradient descent. Experimental results on Gauss-Markov sources with different memory orders demonstrate that our proposed system can match the performance of systems optimized using an exhaustive search.

This paper investigates low-latency adaptive streaming codes for a three-node relay network. A source node transmits a sequence of source packets (messages) to the destination through a relay node. We focus on a particular case where the link connecting the source and relay nodes is almost reliable, but the link connecting the relay to the destination is not. The relay node can observe the erasure pattern that has occurred in the transmission between the source node and itself and adapt its relaying strategy based on that observation. Every source packet must be perfectly recovered by the destination with a strict delay T, as long as the number of erasures in the relay-to-destination link lies below some design parameter. We then characterize capacity as a function of such design parameter. The achievability scheme employs two different relaying strategies, based on whether an erasure has or has not occurred in the link from source to relay. The converse is proven by analyzing a periodic erasure pattern and lower bounding the minimum redundancy across channel packets. We show that the achievable rate can be improved compared to non-adaptive schemes previously proposed, indicating that exploiting the knowledge of the erasure pattern by the relay node is essential in achieving capacity.

This paper studies low-latency streaming codes for the multi-hop network. The source transmits a sequence of messages to a destination through a chain of relays, and requires the destination to reconstruct each message by its deadline. We assume that each communication link is subjected to a certain maximum number of packet erasures. The case of a single relay (a three-node network) was considered in Fong et al. (2020). A coding scheme known as symbol-wise decode and forward was proposed. In the present work, we propose an alternative scheme that is different from Fong et al. (2020) and still achieves the same rate as in Fong et al. (2020) for the one hop case as the field-size goes to infinity. Furthermore, our proposed scheme naturally generalizes to the case of multiple-relay nodes yielding new achievable rates for this setting. The main difference with Fong et al. (2020) is that our proposed scheme exploits the ability of the relay nodes to adapt the transmission based on the erasures on the previous link. Hence, we refer to our scheme as “state-dependent” and contrast it with the scheme in Fong et al. (2020) that is state-independent. Our scheme requires the relay nodes to append a header to the transmitted packets, and we show that the size of the header does not depend on the field-size of the code. We also derive an upper bound on the maximal streaming rate achievable over a network with an arbitrary number of relays. We show that this upper bound matches our achievable rate in the special case when the maximal number of erasures on the first link is greater than or equal to the maximal number of erasures on each of the following links, and the field size goes to infinity.

In the above article[1], an upper bound on the maximum achievable rate is corrected. If we add the restriction that the packets transmitted by the relay must be independent of the erasures introduced by the first-hop channel, then no correction is needed and the converse proof need not be rectified.

We introduce a novel network-adaptive algorithm that is suitable for alleviating network packet losses for low-latency interactive communications between a source and a destination. Our network-adaptive algorithm estimates in real-time the best parameters of a recently proposed streaming code that uses forward error correction (FEC) to correct both arbitrary and burst losses, which cause a crackling noise and undesirable jitters, respectively in audio. In particular, the destination estimates appropriate coding parameters based on its observed packet loss pattern and sends them back to the source for updating the underlying code. Besides, a new explicit construction of practical low-latency streaming codes that achieve the optimal tradeoff between the capability of correcting arbitrary losses and the capability of correcting burst losses is used. Simulation evaluations based on statistical losses and real-world packet loss traces reveal the following: (i) Our proposed network-adaptive algorithm combined with our optimal streaming codes can achieve significantly higher performance compared to uncoded and non-adaptive FEC schemes over UDP (User Datagram Protocol); (ii) Our explicit streaming codes can significantly outperform traditional MDS (maximum-distance separable) streaming schemes when they are used along with our network-adaptive algorithm. In addition, we study different factors that can affect the performance of our network-adaptive algorithm.

Latent variable models have been successfully applied in lossless compression with the bits-back coding algorithm. However, bits-back suffers from an increase in the bitrate equal to the KL divergence between the approximate posterior and the true posterior. In this paper, we show how to remove this gap asymptotically by deriving bits-back coding algorithms from tighter variational bounds. The key idea is to exploit extended space representations of Monte Carlo estimators of the marginal likelihood. Naively applied, our schemes would require more initial bits than the standard bits-back coder, but we show how to drastically reduce this additional cost with couplings in the latent space. When parallel architectures can be exploited, our coders can achieve better rates than bits-back with little additional cost. We demonstrate improved lossless compression rates in a variety of settings, especially in out-of-distribution or sequential data compression.

Applications where multiple users communicate with a common server and desire low latency are common and increasing. This paper studies a network with two source nodes, one relay node and a destination node, where each source nodes wishes to transmit a sequence of messages, through the relay, to the destination, who is required to decode the messages with a strict delay constraint$T$. The network with a single source node has been studied in [1]. We start by introducing two important tools: the delay spectrum, which generalizes delay-constrained point-to-point transmission, and concatenation, which, similar to time sharing, allows combinations of different codes in order to achieve a desired regime of operation. Using these tools, we are able to generalize the two schemes previously presented in [1], and propose a novel scheme which allows us to achieve optimal rates under a set of well-defined conditions. Such novel scheme is further improved in order to achieve higher rates in the scenarios where the conditions for optimality are not met.

We study the problem of transmitting a sequence of messages (streaming messages) through a multi-link, multi-hop packet erasure network. Each message must be reconstructed in-order and under a strict delay constraint. Special cases of our setting with a single link on each hop have been studied recently - the case of a single relay-node, is studied in Fong et al [1]; the case of multiple relays, is studied in Domanovitz et al [2]. As our main result, we propose an achievable rate expression that reduces to previously known results when specialized to their respective settings. Our proposed scheme is based on the idea of concatenating single-link codes from [2] in a judicious manner to achieve the required delay constraints. We propose a systematic approach based on convex optimization to maximize the achievable rate in our framework.

In this paper, we investigate streaming codes over a three-node relay network. Source node transmits a sequence of message packets to the destination via a relay. Source-to-relay and relay-to-destination links are unreliable and introduce at most N1and N2packet erasures, respectively. Destination needs to recover each message packet with a strict decoding delay constraint of T time slots. We propose streaming codes under this setting for all feasible parameters $\{N_{1},\ N_{2},\ T\}$. Relay naturally observes erasure patterns occurring in the source-to-relay link. In our code construction, we employ a channel-state-dependent relaying strategy, which rely on these observations. In a recent work, Fong et al. provide streaming codes featuring channel-state-independent relaying strategies, for all feasible parameters $\{N_{1},\ N_{2},\ T\}$. Our schemes offer a strict rate improvement over the schemes proposed by Fong et al., whenever $N_{1}\lt N_{2}$.

Given the ubiquity of deep neural networks, it is important that these models do not reveal information about sensitive data that they have been trained on. In model inversion attacks, a malicious user attempts to recover the private dataset used to train a supervised neural network. A successful model inversion attack should generate realistic and diverse samples that accurately describe each of the classes in the private dataset. In this work, we provide a probabilistic interpretation of model inversion attacks, and formulate a variational objective that accounts for both diversity and accuracy. In order to optimize this variational objective, we choose a variational family defined in the code space of a deep generative model, trained on a public auxiliary dataset that shares some structural similarity with the target dataset. Empirically, our method substantially improves performance in terms of target attack accuracy, sample realism, and diversity on datasets of faces and chest X-ray images.

In the context of lossy compression, Blau & Michaeli (2019) adopt a mathematical notion of perceptual quality and define the information rate-distortion-perception function, generalizing the classical rate-distortion tradeoff. We consider the notion of universal representations in which one may fix an encoder and vary the decoder to achieve any point within a collection of distortion and perception constraints. We prove that the corresponding information-theoretic universal rate-distortion-perception function is operationally achievable in an approximate sense. Under MSE distortion, we show that the entire distortion-perception tradeoff of a Gaussian source can be achieved by a single encoder of the same rate asymptotically. We then characterize the achievable distortion-perception region for a fixed representation in the case of arbitrary distributions, and identify conditions under which the aforementioned results continue to hold approximately. This motivates the study of practical constructions that are approximately universal across the RDP tradeoff, thereby alleviating the need to design a new encoder for each objective. We provide experimental results on MNIST and SVHN suggesting that on image compression tasks, the operational tradeoffs achieved by machine learning models with a fixed encoder suffer only a small penalty when compared to their variable encoder counterparts.

A typical path over the internet is composed of multiple hops. When considering the transmission of a sequence of messages (streaming messages) through packet erasure channel over a three-node network, it has been shown that taking into account the erasure pattern of each segment can result in improved performance compared to treating the channel as a point-to-point link. Since rarely there is only a single relay between the sender and the destination, it calls for trying to extend this scheme to more than a single relay. In this paper, we first extend the upper bound on the rate of transmission of a sequence of messages for any number of relays. We further suggest an achievable adaptive scheme that is shown to achieve the upper bound up to the size of an additional header that is required to allow each receiver to meet the delay constraints.

The information-theoretic framework of Russo and Zou (2016) and Xu and Raginsky (2017) provides bounds on the generalization error of a learning algorithm in terms of the mutual information between the algorithm's output and the training sample. In this work, we study the proposal, by Steinke and Zakynthinou (2020), to reason about the generalization error of a learning algorithm by introducing a super sample that contains the training sample as a random subset and computing mutual information conditional on the super sample. We first show that these new bounds based on the conditional mutual information are tighter than those based on the unconditional mutual information. We then introduce yet tighter bounds, building on the "individual sample" idea of Bu et al. (2019) and the "data dependent" ideas of Negrea et al. (2019), using disintegrated mutual information. Finally, we apply these bounds to the study of Langevin dynamics algorithm, showing that conditioning on the super sample allows us to exploit information in the optimization trajectory to obtain tighter bounds based on hypothesis tests.

In this work, we consider a sequence of $J$ matrix multiplication jobs which needs to be distributed by a master across multiple worker nodes. For $i\in \{1,2,\ldots,J\}$, job-$i$ begins in round-$i$ and has to be completed by round-$(i+T)$. Previous works consider only the special case of $T=0$ and focus on coding across workers. We propose here two schemes with $T>0$, which feature coding across workers as well as the dimension of time. Our first scheme is a modification of the polynomial coding scheme introduced by Yu et al. and places no assumptions on the straggler model. Exploitation of the temporal dimension helps the scheme handle a larger set of straggler patterns than the polynomial coding scheme, for a given computational load per worker per round. The second scheme assumes a particular straggler model to further improve performance (in terms of encoding/decoding complexity). We develop theoretical results establishing (i) optimality of our proposed schemes for a certain class of straggler patterns and (ii) improved performance for the case of i.i.d. stragglers. These are further validated by experiments, where we implement our schemes to train neural networks.

This paper investigates low-latency streaming codes for a three-node relay network. The source transmits a sequence of messages (streaming messages) to the destination through the relay between them, where the first-hop channel from the source to the relay and the second-hop channel from the relay to the destination are subject to packet erasures. Every source message generated at a time slot must be recovered perfectly at the destination within the subsequent T time slots. In any sliding window of $ {T}+1$ time slots, we assume no more than $ {N}_{1}$ and ${N}_{2}$ erasures are introduced by the first-hop channel and second-hop channel respectively. We fully characterize the maximum achievable rate in terms of T, $ {N}_{1}$ and $ {N}_{2}$ . The achievability is proved by using a symbol-wise decode-forward strategy where the source symbols within the same message are decoded by the relay with different delays. The converse is proved by analyzing the maximum achievable rate for each channel when the erasures in the other channel are consecutive (bursty). In addition, we show that traditional message-wise decode-forward strategies, which require the source symbols within the same message to be decoded by the relay with the same delay, are sub-optimal in general.

This paper considers multiplexing two sequences of messages with two different decoding delays over a packet erasure channel. In each time slot, the source constructs a packet based on the current and previous messages and transmits the packet, which may be erased when the packet travels from the source to the destination. The destination must perfectly recover every source message in the first sequence subject to a decoding delay Tv, and every source message in the second sequence subject to a shorter decoding delay Tu≤ Tv,. We assume that the channel loss model introduces a burst erasure of a fixed length B on the discrete timeline. Under this channel loss assumption, the capacity region for the case where Tv, ≤ Tu+B was previously solved. In this paper, we fully characterize the capacity region for the remaining case Tv, ) Tu+B. The key step in the achievability proof is achieving the non-trivial corner point of the capacity region through using a multiplexed streaming code constructed by superimposing two single-stream codes. The main idea in the converse proof is obtaining a genie-aided bound when the channel is subject to a periodic erasure pattern where each period consists of a length-B burst erasure followed by a length-Tunoiseless duration.

In this paper, we consider transmitting a sequence of messages (a streaming source) over a packet erasure channel, where every source message must be recovered perfectly at the destination subject to a fixed decoding delay. Recently, the capacity of such a channel was established. However, the codes shown to achieve the capacity are either non-explicit constructions (proven to exist) or explicit constructions requiring large field size that scales exponentially with the delay. This work presents an explicit rate-optimal construction for all channel and delay parameters over a field size that scales only quadratically with the delay.

This paper presents a new construction of error correcting codes which achieves optimal recovery of a streaming source over a packet erasure channel. The channel model considered is the sliding-window erasure model, with burst and arbitrary losses, introduced by Badr et al. We present a simple construction, when the rate of the code is at least 1/2, which achieves optimal error correction in this setup. Our proposed construction is explicit and systematic. It uses off-the-shelf maximum distance separable (MDS) codes and maximum rank distance (MRD) Gabidulin block codes as constituent codes and combines them in a simple manner. This is in contrast to other recent works, where the construction involves a careful design of the generator or parity check matrix from first principles. The field size requirement which depends on the constituent MDS and MRD codes is also analyzed.

Motivated by real-world machine learning applications, we consider a statistical classification task in a sequential setting where test samples arrive sequentially. In addition, the generating distributions are unknown and only a set of empirically sampled sequences are available to a decision maker. The decision maker is tasked to classify a test sequence which is known to be generated according to either one of the distributions. In particular, for the binary case, the decision maker wishes to perform the classification task with minimum number of the test samples, so, at each step, she declares that either hypothesis 1 is true, hypothesis 2 is true, or she requests for an additional test sample. We propose a classifier and analyze the type-I and type-II error probabilities. We demonstrate the significant advantage of our sequential scheme compared to an existing non-sequential classifier proposed by Gutman. Finally, we extend our setup and results to the multi-class classification scenario and again demonstrate that the variable-length nature of the problem affords significant advantages as one can achieve the same set of exponents as Gutman's fixed-length setting but without having the rejection option.

We introduce a novel network-adaptive algorithm that is suitable for alleviating network packet losses for low-latency interactive communications between a source and a destination. Network packet losses happen in a bursty manner as well as an arbitrary manner, where the former is usually due to network congestion and the latter can be caused by unreliable wireless links. Our network-adaptive algorithm estimates in real time the best parameters of a recently proposed streaming code that corrects both arbitrary losses (which cause crackling noise in audio) and burst losses (which cause undesirable jitters and pauses in audio) using forward error correction (FEC). The network-adaptive algorithm updates the coding parameters in real time as follows: The destination estimates appropriate coding parameters based on its observed packet loss pattern and then the parameters are fed back to the source for updating the underlying code. In addition, a new explicit construction of practical low-latency streaming codes that achieve the optimal tradeoff between the capability of correcting arbitrary losses and the capability of correcting burst losses is provided. Simulation evaluations based on real-world packet loss traces reveal that our proposed network-adaptive algorithm combined with our optimal streaming codes achieves significantly higher reliability compared to uncoded and non-adaptive FEC schemes over UDP (User Datagram Protocol).

In this work, we improve upon the stepwise analysis of noisy iterative learning algorithms initiated by Pensia, Jog, and Loh (2018) and recently extended by Bu, Zou, and Veeravalli (2019). Our main contributions are significantly improved mutual information bounds for Stochastic Gradient Langevin Dynamics via data-dependent estimates. Our approach is based on the variational characterization of mutual information and the use of data-dependent priors that forecast the mini-batch gradient based on a subset of the training samples. Our approach is broadly applicable within the information-theoretic framework of Russo and Zou (2015) and Xu and Raginsky (2017). Our bound can be tied to a measure of flatness of the empirical risk surface. As compared with other bounds that depend on the squared norms of gradients, empirical investigations show that the terms in our bounds are orders of magnitude smaller.

In smart metering systems, a rechargeable battery can be utilized to protect the privacy of a user from the utility provider by partially masking the load profile of the user. In this line of research on using rechargeable batteries for privacy protection, most existing works have studied only single-user systems using rechargeable batteries. In this letter, we consider a multi-user scenario where the power supplies of two or more users are combined before sending them to the utility provider. We study the effect of such a user cooperation on enhancing the user privacy by deriving upper and lower bounds on the minimum leakage rate. Our simulation results show that the information leakage of each user can be reduced by a factor of the total number of cooperative users.

Regenerating codes are efficient methods for distributed storage in storage networks, where node failures are common. They guarantee low cost data reconstruction and repair through accessing only a predefined number of arbitrarily chosen storage nodes in the network. In this paper, we consider two simultaneous extensions to the original regenerating codes framework introduced by Dimakis et al.; 1) both data reconstruction and repair are resilient to the presence of a certain number of erroneous nodes in the network and 2) the number of helper nodes in every repair is not fixed, but is a flexible parameter that can be selected during the run-time. We study the fundamental limits of required total repair bandwidth and provide an upper bound for the storage capacity of these codes under these assumptions. We then focus on the minimum repair bandwidth (MBR) case and derive the exact storage capacity by presenting explicit coding schemes with exact repair, which achieve the upper bound of the storage capacity in the considered setup. To this end, we first provide a more natural extension of the well-known product matrix (PM) MBR codes, modified to provide flexibility in choosing the number of helpers in each repair, and simultaneously be robust to erroneous nodes in the network. This is achieved by proving the non-singularity for a family of matrices in large enough finite fields. We next provide another extension of the PM codes, based on a novel repair scheme which enables flexibility in the number of helpers and robustness against erroneous nodes without any extra cost in field size compared with the original PM codes.

This paper considers transmitting a sequence of messages (streaming messages) over a packet erasure channel. In each time slot, the source constructs a packet based on the current and the previous messages and transmits the packet, which may be erased when the packet travels from the source to the destination. Every source message must be recovered perfectly at the destination subject to a fixed decoding delay. We assume that the channel loss model introduces either one burst erasure or multiple arbitrary erasures in any fixed-sized sliding window. Under this channel loss assumption, we fully characterize the maximum achievable rate by constructing streaming codes that achieve the optimal rate. In addition, our construction of optimal streaming codes implies the full characterization of the maximum achievable rate for convolutional codes with any given column distance, column span, and decoding delay. Numerical results demonstrate that the optimal streaming codes outperform existing streaming codes of comparable complexity over some instances of the Gilbert-Elliott channel and the Fritchman channel.

The unprecedented growth in the number of mobile data and connected machines ever-fast approaches limits of fourth generation technologies to address this enormous data demand. Therefore, the development of the fifth generation (5G) wireless communication technologies is a priority issue currently. The evolution towards 5G wireless communications will be a cornerstone for realizing the future human-centric and connected machine-centric networks, which achieve near-instantaneous, zero distance connectivity for people and connected machines. On the other hand, wireless networks have been widely used in civilian and military applications and become an indispensable part of our daily life. People rely heavily on wireless networks for transmission of important/private information, such as credit card information, energy pricing, e-health data, command, and control messages. Therefore, security is a critical issue for future 5G wireless networks. Physical layer security techniques can be used to either perform secure data transmission directly or generate the distribution of cryptography keys for conventional cryptography techniques in the 5G networks. With careful management and implementation, physical layer security can be used as an additional level of protection on top of the existing security schemes. As such, they will formulate a well-integrated security solution together that efficiently safeguards the confidential and privacy communication data in 5G wireless networks. The main goal of this IEEE JSAC Special Issue on “Physical Layer Security for 5G Wireless Networks” is to bring together leading researchers in both academia and industry from diversified backgrounds to advance the theory and practice of physical layer security for 5G wireless networks.

Physical layer security which safeguards data confidentiality based on the information-theoretic approaches has received significant research interest recently. The key idea behind physical layer security is to utilize the intrinsic randomness of the transmission channel to guarantee the security in physical layer. The evolution toward 5G wireless communications poses new challenges for physical layer security research. This paper provides a latest survey of the physical layer security research on various promising 5G technologies, including physical layer security coding, massive multiple-input multiple-output, millimeter wave communications, heterogeneous networks, non-orthogonal multiple access, full duplex technology, and so on. Technical challenges which remain unresolved at the time of writing are summarized and the future trends of physical layer security in 5G and beyond are discussed.

The unprecedented growth in the number of mobile data and connected machines ever-fast approaches limits of fourth generation technologies to address this enormous data demand. Therefore, the development of the fifth generation (5G) wireless communication technologies is a priority issue currently. The evolution towards 5G wireless communications will be a cornerstone for realizing the future human-centric and connected machine-centric networks, which achieve near-instantaneous, zero distance connectivity for people and connected machines. On the other hand, wireless networks have been widely used in civilian and military applications and become an indispensable part of our daily life. People rely heavily on wireless networks for transmission of important/private information, such as credit card information, energy pricing, e-health data, command, and control messages. Therefore, security is a critical issue for future 5G wireless networks. Physical layer security techniques can be used to either perform secure data transmission directly or generate the distribution of cryptography keys for conventional cryptography techniques in the 5G networks. With careful management and implementation, physical layer security can be used as an additional level of protection on top of the existing security schemes. As such, they will formulate a well-integrated security solution together that efficiently safeguards the confidential and privacy communication data in 5G wireless networks. The main goal of this IEEE JSAC Special Issue on “Physical Layer Security for 5G Wireless Networks” is to bring together leading researchers in both academia and industry from diversified backgrounds to advance the theory and practice of physical layer security for 5G wireless networks.

The problem of secure broadcasting with independent secret keys is studied. The particular scenario is analyzed in which a common message has to be broadcast to two legitimate receivers, while keeping an external eavesdropper ignorant of it. The transmitter shares independent secret keys of sufficiently high rates with both legitimate receivers, which can be used in different ways: they can be used as one-time pads to encrypt the common message, as fictitious messages for wiretap coding, or as a hybrid of these. In this paper, capacity results are established when the broadcast channels involving the three receivers are degraded. If both legitimate channels are degraded versions of the eavesdropper's channel, it is shown that the one-time pad approach is optimal for several cases, yielding corresponding capacity expressions. Alternatively, the wiretap coding approach is shown to be optimal if the eavesdropper's channel is degraded with respect to both legitimate channels, establishing capacity in this case as well. If the eavesdropper's channel is neither the strongest nor the weakest, an intricate scheme that carefully combines both concepts of one-time pad and wiretap coding with fictitious messages turns out to be capacity-achieving. Finally we also obtain some results for the general non-degraded broadcast channel.

We consider the problem of covert communication over a state-dependent channel, where the transmitter has causal or noncausal knowledge of the channel states. Here, covert means that a warden on the channel should observe similar statistics when the transmitter is sending a message and when it is not. When a sufficiently long secret key is shared between the transmitter and the receiver, we derive closed-form formulas for the maximum achievable covert communication rate (covert capacity) for discrete memoryless channels and, when the transmitter's channel-state information (CSI) is noncausal, for additive white Gaussian noise (AWGN) channels. For certain channel models, including the AWGN channel, we show that the covert capacity is positive with CSI at the transmitter, but is zero without CSI. We also derive lower bounds on the rate of the secret key that is needed for the transmitter and the receiver to achieve the covert capacity.

We consider a communication setup where two source streams with different decoding deadlines, must be simultaneously transmitted over a single channel subjected to burst erasures. The encoder multiplexes the two source streams into a single stream of channel-packets. The decoder must recover the two source streams sequentially by their corresponding deadlines. One of the streams, the urgent stream, has a smaller delay than the other stream. We study the capacity region for such a setting for a certain range of system parameters. We divide the system into three different cases based on the relative values of the delays. For each case we provide achievability and converse bounds, which match under certain conditions. Our proposed coding scheme involves a careful construction of the parity check packets by jointly coding across the two streams despite different deadlines. Interestingly it is possible to transmit the urgent stream at a certain positive rate even when the sum rate equals the capacity associated with the less urgent stream. A separation based approach where we apply separate single-stream codes to each stream is suboptimal. Although our capacity results assume a simplistic channel model with a single erasure burst, we further demonstrate that our proposed code constructions also provide significant performance gains in simulations over statistical channel models with random bursts.

The problem of sending a secret message over the Gaussian multiple-input multiple-output (MIMO) wiretap channel is studied. While the capacity of this channel is known, it is not clear how to construct optimal coding schemes that achieve this capacity. In this paper, we use linear operations along with successive interference cancellation to attain effective parallel single-antenna wiretap channels. By using independent scalar Gaussian wiretap codebooks over the resulting parallel channels, the capacity of the MIMO wiretap channel is achieved. The derivation of the schemes is based upon joint triangularization of the channel matrices. We find that the same technique can be used to rederive capacity expressions for the MIMO wiretap channel in a way that is simple and closely connected to a transmission scheme. This technique allows to extend the previously proven strong security for scalar Gaussian channels to the MIMO case. We further consider the problem of transmitting confidential messages over a two-user broadcast MIMO channel. For that problem, we find that derivation of both the capacity and a transmission scheme is a direct corollary of the proposed analysis for the MIMO wiretap channel.

In this paper, we study a diamond-relay channel where the source is connected toMrelays through orthogonal links and the relays transmit to the destination over a wireless multiple-access channel in the presence of an eavesdropper. The eavesdropper not only observes the relay transmissions through another multiple-access channel but also observes a certain number of source-relay links. The legitimate terminals know neither the eavesdropper's channel state information nor the location of source-relay links revealed to the eavesdropper except the total number of such links. For this wiretapped diamond-relay channel, we establish the optimal secure d.o.f. In the achievability part, our proposed scheme uses the source-relay links to transmit a judiciously constructed combination of message symbols, artificial noise symbols, and fictitious message symbols associated with secure network coding. The relays use a combination of beamforming and interference alignment in their transmission scheme. For the converse part, we take a genie-aided approach assuming that the location of wiretapped links is known.

Smart-metering systems report electricity usage of a user to the utility provider on almost real-time basis. This could leak private information about the user to the utility provider. In this paper, we investigate the use of a rechargeable battery in order to provide privacy to the user. We assume that the user load sequence is a first-order Markov process, the battery satisfies ideal charge conservation, and that privacy is measured using normalized mutual information (leakage rate) between the user load and the battery output. We study the optimal battery charging policy that minimizes the leakage rate among the class of battery policies that satisfy causality and charge conservation. We propose a series reduction on the original problem and ultimately recast it as a Markov Decision Process (MDP) that can be solved using a dynamic program. In the special case of i.i.d. demand, we explicitly characterize the optimal policy and show that the associated leakage rate can be expressed as a single-letter mutual information expression. In this case, we show that the optimal charging policy admits an intuitive interpretation of preserving a certain invariance property of the state. Interestingly an alternative proof of optimality can be provided that does not rely on the MDP approach, but is based on purely information theoretic reductions.

In the distributed storage systems (DSSs) with k systematic nodes, robustness against node failure is commonly provided by storing redundancy in a number of other nodes and performing repair mechanism to reproduce the content of the failed nodes. Efficiency is then achieved by minimizing the storage overhead and the amount of data transmission required for data reconstruction and repair, provided by coding solutions, such as regenerating codes. Common explicit regenerating code constructions enable efficient repair through accessing a predefined number, d, of arbitrary chosen available nodes, namely helpers. In practice, however, the state of the system dynamically changes based on the request load, the link traffic, and so on, and the parameters which optimize system's performance vary accordingly. It is then desirable to have coding schemes which are able to operate optimally under a range of different parameters simultaneously. Specifically, adaptivity in the number of helper nodes for repair is of interest. While robustness requires capability of performing repair with small number of helpers, it is desirable to use as many helpers as available to reduce the transmission delay and total repair traffic. In this paper, we focus on the minimum storage regenerating (MSR) codes, where each of the n nodes in the network is supposed to store α information units, and the source data of size kα could be recovered from any arbitrary set of k nodes. We introduce a class of MSR codes that realize the optimal repair bandwidth simultaneously with a set of different choices for the number of helpers, namely D = {d1, . . . , dδ}. Our coding scheme follows the product matrix (PM) framework introduced by Rashmi et al. and could be considered as a generalization of the PM MSR code presented by Rashmi et al., such that any di= (i + 1)(k - 1) helpers can perform an optimal repair. As a result, the coding rate in our construction is limited by (k/n) ≤ (1/2). However, similar to the original design of PM MSR codes, our solution can realize practical values of the parameter α. Recently, Ye and Barg have presented another explicit MSR coding scheme which is capable of performing optimal repair for various number of helpers. The solution presented by Ye and Barg works for any arbitrary set of parameters k and D and can achieve high-coding rates, but the required α for this code is exponentially large. We show that the required value for α in the coding scheme presented in this paper is exponentially smaller when compared with the work of Ye and Barg for the same set of other parameters. Particularly, for a DSS with n nodes and k systematic nodes, the required value for α is reduced from sn to sk, where s = 1 cm(d1-k+1, ⋯, dδ-k+1). We also show the required field size in the presented coding scheme is equal to n.

The burst rank loss network is an extension of the burst erasure channel, where the channel matrix between the sender and receiver becomes rank-deficient for a certain period of consecutive time-slots. We study streaming communication over the burst rank loss channel, propose a new class of codes, ROBIN codes, and establish their optimality. Our construction uses the maximum rank distance and maximum sum rank codes from previous works as constituent codes and combines them in a layered fashion. Our results generalize previous works on both the single-link and multiple-parallel-link streaming setups over burst erasure channels. We perform simulations over statistical network models to show that ROBIN codes attain low packet loss rates in comparison with the existing codes.

We introduce a new class of forward error correction (FEC) codes for VoIP communications which support different recovery delay depending on the channel conditions. Specifically, our proposed class of Dual-Delay (DD) codes can recover from challenging long bursts of losses with close to theoretical minimum delay so as to meet playback deadlines for recovered packets. They further improve conversational interactivity by achieving lower recovery delay during periods of random isolated losses. These DD codes are shown to achieve lower residual loss rates when compared to existing codes over a wide range of parameters of the Gilbert-Elliott channel. Experiments over real world packet traces further show performance gains of DD codes in terms of perceptually motivated ITU-T G.107 E-model.

We study a setting where two source streams with different decoding deadlines must be transmitted over a burst erasure channel. The source streams are multiplexed into a single stream of channel-packets at the encoder and transmitted over a packet erasure channel. The decoder must recover the source-packets within each stream sequentially, by their corresponding deadlines. We consider the burst-erasure channel model and characterize the capacity region for a certain range of system parameters. We show that the operation of the system can be divided into three different regimes based on the relative values of decoding deadlines. On the achievability side, we show that jointly coding across the two streams, despite their different deadlines, is necessary to achieve capacity. On the converse side we develop information theoretic outer bounds on the capacity region. We find that the capacity region exhibits a “corner point” where we can transmit the urgent stream at a positive rate, yet attain a sum-rate equal to the capacity of the non-urgent stream.

The sum-rate distortion function of generalized Gaussian multiterminal source coding is shown to coincide with that of joint encoding in the high-resolution regime if and only if the source-encoder bipartite graph and the undirected graphical model (also known as Gaussian Markov network or Gaussian Markov random field) of the source distribution satisfy a certain condition.

In this paper, a streaming transmission setup is considered, where an encoder observes a new message in the beginning of each block and a decoder sequentially decodes each message after a delay of T blocks. In this streaming setup, the fundamental interplay between the coding rate, the error probability, and the blocklength in the moderate deviations regime is studied. For output symmetric channels, the moderate deviations constant is shown to improve over the block coding or non-streaming setup by exactly a factor of T for a certain range of moderate deviations scalings. For the converse proof, a more powerful decoder, to which some extra information is fedforward is assumed. The error probability is bounded first for an auxiliary channel and this result is translated back to the original channel by using a newly developed change-of-measure lemma, where the speed of decay of the remainder term in the exponent is carefully characterized. For the achievability proof, a known coding technique that involves a joint encoding and decoding of fresh and past messages is applied with some manipulations in the error analysis.

We consider the problem of covert communication over a state-dependent channel, where the transmitter has non-causal knowledge of the channel states. Here, “covert” means that the probability that a warden on the channel can detect the communication must be small. In contrast with traditional models without noncausal channel-state information at the transmitter, we show that covert communication can be possible with positive rate. We derive closed-form formulas for the maximum achievable covert communication rate (“covert capacity”) in this setting for discrete memoryless channels as well as additive white Gaussian noise channels. We also derive lower bounds on the rate of the secret key that is needed for the transmitter and the receiver to achieve the covert capacity.

We consider the problem of sequential transmission of Gauss-Markov sources. We show that in the limit of large spatial block lengths, greedy compression with respect to the squared error distortion is optimal; that is, there is no tension between optimizing the distortion of the source in the current time instant and that of future times. We then extend this result to the case where at time t a random compression rate rtis allocated independently of the rate at other time instants. This, in turn, allows us to derive the optimal performance of sequential coding over packet-erasure channels with instantaneous feedback. For the case of packet erasures with delayed feedback, we connect the problem to that of compression with side information that is known at the encoder and may be known at the decoder - where the most recent packets serve as side information that may have been erased, and demonstrate that the loss due to a delay by one time unit is rather small.

In coding for distributed storage systems, efficient data reconstruction and repair through accessing a predefined number of arbitrarily chosen storage nodes is guaranteed by regenerating codes. Traditionally, code parameters, specially the number of helper nodes participating in a repair process, are predetermined. However, depending on the state of the system and network traffic, it is desirable to adapt such parameters accordingly in order to minimize the cost of repair. In this work a class of regenerating codes with minimum storage is introduced that can simultaneously operate at the optimal repair bandwidth, for a wide range of exact repair mechanisms, based on different number of helper nodes.

A rechargeable battery may alleviate the issue of privacy loss in a smart metering system by distorting a household's load profile. However, existing studies involve a single rechargeable battery, whereas in a network scenario, there could be multiple batteries connected together. In this letter, we study the extension where a user's electricity load is input into a network of two rechargeable batteries, connected in series, and operating individually. This battery network attempts to mask the user load from the utility provider. We focus on the case of independent identically distributed load profile and a system of ideal batteries with no conversion loss, and use normalized mutual information (leakage rate) as the privacy metric. We derive upper and lower bounds on the leakage rate in terms of (single-letter) mutual information expressions. On the achievability side, our information-theoretic upper bound captures the novel tension between minimizing the leakage across each individual battery and the effect of their joint interaction. For the lower bound, we show that a system with a single battery, whose storage capacity is the sum of the two individual batteries, can achieve a leakage rate at least as small as our proposed setup. Furthermore, we use simulations to compare achievable leakage of our proposed scheme with several baseline schemes. The achievable leakage rates obtained in this study could help us to elucidate the privacy performance of a network of batteries.

We study error correction codes for multimedia streaming applications where a stream of source packets must be transmitted in real-time, with in-order decoding, and strict delay constraints. In our setup, the encoder observes a stream of source packets in a sequential fashion, and M channel packets must be transmitted between the arrival of successive source packets. Each channel packet can depend on all the source packets observed up to and including that time, but not on any future source packets. The decoder must reconstruct the source stream with a delay of T packets. We consider a class of packet erasure channels with burst and isolated erasures, where the erasure patterns are locally constrained. Our proposed model provides a tractable approximation to statistical models, such as the Gilbert-Elliott channel, for capacity analysis. When M = 1, i.e., when the source-packet arrival and channel-packet transmission rates are equal, we establish upper and lower bounds on the capacity, that are within one unit of the decoding delay T. We also establish necessary and sufficient conditions on the column distance and column span of a convolutional code to be feasible, and in turn establish a fundamental tradeoff between these. Our proposed codes-maximum distance and span codes- achieve a near-optimal tradeoff between the column distance and column span, and involve a layered construction. When M > 1, we establish the capacity for the burst-erasure channel and an achievable rate in the general case. Extensive numerical simulations over Gilbert-Elliott and Fritchman channel models suggest that our codes also achieve significant gains in the residual loss probability over statistical channel models.

We propose a new coding technique for sequential transmission of a stream of Gauss-Markov sources over erasure channels under a zero decoding delay constraint. Our proposed scheme is a combination (hybrid) of predictive coding with truncated memory, and quantization-and-binning. We study the optimality of our proposed scheme using an information theoretic model. In our setup, the encoder observes a stream of source vectors that are spatially independent and identically distributed (i.i.d.) and temporally sampled from a first-order stationary Gauss-Markov process. The channel introduces an erasure burst of a certain maximum length B, starting at an arbitrary time, not known to the transmitter. The reconstruction of each source vector at the destination must be with zero delay and satisfy a quadratic distortion constraint with an average distortion of D. The decoder is not required to reconstruct those source vectors that belong to the period spanning the erasure burst and a recovery window of length W following it. We study the minimum compression rate R(B, W, D) in this setup. As our main result, we establish upper and lower bounds on the compression rate. The upper bound (achievability) is based on our hybrid scheme. It achieves significant gains over baseline schemes such as (leaky) predictive coding, memoryless binning, a separation-based scheme, and a group of pictures-based scheme. The lower bound is established by observing connection to a network source coding problem. The bounds simplify in the high resolution regime, where we provide explicit expressions whenever possible, and identify conditions when the proposed scheme is close to optimal. We finally discuss the interplay between the parameters of our burst erasure channel and the statistical channel models and explain how the bounds in the former model can be used to derive insights into the simulation results involving the latter. In particular, our proposed scheme outperforms the baseline schemes over the i.i.d. erasure channel and the Gilbert-Elliott channel, and achieves performance close to a lower bound in some regimes.

It is well known that dispersed and burst packet losses introduce significantly different amount of distortions. Since perceptual models are typically content dependent, it is challenging to characterize how losses interact with concealment. This paper presents loss-pattern-aware distortion (LoPAD), a content-independent metric that explicitly models the impact of different loss patterns. LoPAD operates solely on the loss trace without analyzing received media. It is fast, and supports offline and cloud-based monitoring of network impairment. Taking audio conferencing as target application, we show that full-reference PESQ scores for a collection of speech samples can be closely approximated by LoPAD. For various combinations of erasure channel models and forward error correction (FEC) codes, the correlation coefficients between LoPAD and PESQ-DMOS range from 0.90 to 0.97.

We consider streaming data transmission over a discrete memoryless channel. A new message is given to the encoder at the beginning of each block and the decoder decodes each message sequentially, after a delay of T blocks. In this streaming setup, we study the fundamental interplay between the rate and error probability in the central limit and moderate deviations regimes and show that: 1) in the moderate deviations regime, the moderate deviations constant improves over the block coding or non-streaming setup by a factor of T and 2) in the central limit regime, the second-order coding rate improves by a factor of approximately √T for a wide range of channel parameters. For both the regimes, we propose coding techniques that incorporate a joint encoding of fresh and previous messages. In particular, for the central limit regime, we propose a coding technique with truncated memory to ensure that a summation of constants, which arises as a result of applications of the central limit theorem, does not diverge in the error analysis. Furthermore, we explore interesting variants of the basic streaming setup in the moderate deviations regime. We first consider a scenario with an erasure option at the decoder, i.e., the decoder can output an erasure symbol instead of a message estimate, and show that both the exponents of the total error and the undetected error probabilities improve by factors of T. Next, by utilizing the erasure option, we show that the exponent of the total error probability can be improved to that of the undetected error probability (in the order sense) at the expense of a variable decoding delay.

In this paper, we consider the Gaussian diamond-wiretap channel that consists of an orthogonal broadcast channel from a source to two relays and a Gaussian fast-fading multiple access-wiretap channel from the two relays to a legitimate destination and an eavesdropper. For the multiple access part, we consider both the case with full channel state information (CSI) and the case with no eavesdropper's CSI, at the relays and the legitimate destination. For both the cases, we establish the exact secure degrees of freedom and generalize the results for multiple relays. For the converse part, we introduce a new technique of capturing the trade-off between the message rate and the amount of individual randomness injected at each relay. In the achievability part, we show (i) how to strike a balance between sending message symbols and common noise symbols from the source to the relays in the broadcast component and (ii) how to combine artificial noise-beamforming and noise-alignment techniques at the relays in the multiple access component.

Regenerating codes are efficient methods for distributed storage in practical networks where node failures are common. They guarantee low cost data reconstruction and repair through accessing only a predefined number of arbitrary chosen storage nodes in the network. In this work we study the fundamental limits of required total repair bandwidth and the storage capacity of these codes under the assumption that i) both data reconstruction and repair are resilient to the presence of a certain number of erroneous nodes in the network and ii) the number of helper nodes in every repair is not fixed, but is a flexible parameter that can be selected during the run-time. We focus on the minimum repair bandwidth point in this work, propose the associated coding scheme to posses both these extra properties, and prove its optimality.

In the classic burst erasure channel, packets are consecutively erased in bursts between gaps of perfect communication. For the Burst Rank Loss Network, instead of a single-link erasure channel, there is a channel matrix that is rank-deficient for a burst of time before returning to full-rank. We establish the streaming capacity of the Burst Rank Loss Network and construct a new family of layered codes referred to as Recovery Of Bursts In Networks (ROBIN) codes that achieve capacity. Our results generalize previous work on both the single-link and multiple-parallel-link streaming setups. Simulations over statistical network models show that ROBIN codes lose fewer packets than baseline codes.

This paper investigates the two-user state-dependent Gaussian multiple access channel (MAC) with a helper. The channel is corrupted by an additive Gaussian state sequence known to neither the transmitters nor the receiver, but to a helper noncausally, which assists state cancellation at the receiver. Inner and outer bounds on the capacity region are first derived, which improve the previous bounds given by Duan et al. Further comparison of these bounds yields either segments on the capacity region boundary or the full capacity region by considering various cases of channel parameters.

Motivated by recent interest in physical-layer secret-key generation over wireless fading channels, we study the non-coherent secret-key generation capacity of a block-fading wireless channel with channel reciprocity and bi-directional (two-way) communication. We assume a non-coherent main channel, i.e., the realization of channel gains on the main channel is not known to any terminal. The eavesdropper is assumed to have both perfect Channel State Information of its own channel and orthogonal observations from the forward and backward channels. As our main result, we establish new upper and lower bounds on the secret-key generation capacity with public discussion, which are structurally similar. The upper bound can be expressed as a sum of three terms-one of the terms arises due to channel reciprocity, while the other two terms correspond to communication on the forward and backward channels, respectively. In the limit of long coherence period, the contribution from channel reciprocity vanishes to zero, whereas the other terms prevail. The lower bound is based on a separation based scheme. In each coherence block, we use the first symbol for training while the remainder of the coherence block is devoted to source emulation, i.e., to generate correlated sources between the terminals. The lower bound expression also consists of three terms and admits an interpretation similar to the upper bound expression. For Rayleigh fading channels, in the high signal-to-noise ratio (SNR) regime, the gap between the upper and lower bounds is shown to vanish inversely with the coherence period. Numerical results indicate significant performance gains over training-only schemes even for moderate values of SNR and small coherence periods.

The column Hamming distance of a convolutional code determines the error correction capability when streaming over a class of packet erasure channels. We introduce a metric known as the column sum rank that parallels the column Hamming distance when streaming over a network with link failures. We prove the rank analogues of several known column Hamming distance properties and introduce a new family of convolutional codes that maximize the column sum rank up to the code memory. Our construction involves finding a class of super-regular matrices that preserve this property after multiplication with non-singular block diagonal matrices in the ground field.

The state-dependent point-to-point Gaussian channel with a helper is first studied, in which a transmitter communicates with a receiver via a state-corrupted channel. The state is not known to the transmitter nor to the receiver, but known to a helper noncausally, which then wishes to assist the receiver to cancel the state. Differently from the previous work that characterized the capacity only in the infinite state power regime, this paper explores the general case with arbitrary state power. A lower bound on the capacity is derived based on an achievable scheme that integrates direct state subtraction and single-bin dirty paper coding. By analyzing this lower bound and further comparing it with the existing upper bounds, the capacity of the channel is characterized for a wide range of channel parameters. Such an idea of characterizing the capacity is further extended to study the two-user state-dependent multiple access channel with a helper. By comparing the derived inner and outer bounds, the channel parameters are partitioned into appropriate cases, and for each case, either segments on the capacity region boundary or the full capacity region are characterized.

Motivated by error correction coding in multimedia applications, we study the problem of broadcasting a single common source to multiple receivers over heterogeneous erasure channels. Each receiver is required to partially reconstruct the source sequence by decoding a certain fraction of the source symbols. We propose a coding scheme that requires only off-the-shelf erasure codes and can be easily adapted as users join and leave the network. Our scheme involves splitting the source sequence into multiple segments and applying a systematic erasure code to each such segment. We formulate the problem of minimizing the transmission latency at the server as a linear programming problem and explicitly characterize an optimal choice for the code rates and segment sizes. Through numerical comparisons, we demonstrate that our proposed scheme outperforms both separation-based coding schemes and degree-optimized rateless codes and performs close to a natural outer (lower) bound in certain cases. We further study individual user decoding delays for various orderings of segments in our scheme. We provide closed-form expressions for each individual user's excess latency when parity checks are successively transmitted in both increasing order and decreasing order of their segment's coded rate and also qualitatively discuss the merits of each order.

We investigate the problem of secure broadcasting over fast fading channels with imperfect main channel state information (CSI) at the transmitter. In particular, we analyze the effect of the noisy estimation of the main CSI on the throughput of a broadcast channel where the transmission is intended for multiple legitimate receivers in the presence of an eavesdropper. Besides, we consider the realistic case where the transmitter is only aware of the statistics of the eavesdropper's CSI and not of its channel's realizations. First, we discuss the common message transmission case where the source broadcasts the same information to all the receivers, and we provide an upper and a lower bound on the ergodic secrecy capacity. For this case, we show that the secrecy rate is limited by the legitimate receiver having, on average, the worst main channel link and we prove that a nonzero secrecy rate can still be achieved even when the CSI at the transmitter is noisy. Then, we look at the independent messages case where the transmitter broadcasts multiple messages to the receivers, and each intended user is interested in an independent message. For this case, we present an expression for the achievable secrecy sum-rate and an upper bound on the secrecy sum-capacity and we show that, in the limit of large number of legitimate receivers K, our achievable secrecy sum-rate follows the scaling law log ((1-α)log(K)), where α is the estimation error variance of the main CSI. The special cases of high SNR, perfect and no-main CSI are also analyzed. Analytical derivations and numerical results are presented to illustrate the obtained expressions for the case of independent and identically distributed Rayleigh fading channels.

We study low-delay streaming codes for erasure channels in point-to-point and multicast scenarios. We consider a sliding window erasure channel which captures the temporal correlation in packet losses observed in real channels. This correlation is often modelled using statistical channels such as Gilbert-Elliott channel. In the point-to-point case, we provide a new class of codes, Embedded Maximum Distance Separable (EMDS) codes, which recovers from channels introducing a mixture of burst and isolated erasures. Moreover, we propose a technique that extends point-to-point codes for the multicast scenario with two receivers that tolerate different delays, T1and T2. The multicast codes opportunistically decode packets with short delay T1when the channel is relatively better and with long delay T2when the channel is worse. Simulations over multicast Gilbert-Elliott channels show that EMDS codes outperform other streaming codes for both users.

Two setups involving delay-constrained sequential transmission of a vector Gauss-Markov source over a burst-erasure channel are studied. The encoder sequentially compresses the source vectors to be transmitted in a causal fashion. The channel introduces a single erasure burst of length up to B during the transmission. In streaming with controlled-interruption, the decoder reconstructs the source vectors within average distortion D and maximum delay of T, whenever the channel packets are not erased. In streaming with ideal-playback, the decoder reconstructs all the source vectors within average distortion D and maximum delay of T. Upper and lower bounds on the minimum compression rate are derived for each setup. The bounds coincide in the high resolution regime for both cases and in large delay regime for the case of ideal-playback.

We study sequential transmission of Gauss-Markov sources over erasure channels under a zero decoding delay constraint. A two-stage coding scheme which can be described as a hybrid between predictive coding with limited past and quantization & binning is proposed. This scheme can achieve significant performance gains over baseline schemes in simulations involving i.i.d. erasure channels, and in certain regimes can attain performance close to a fundamental lower bound. We consider an information theoretic model for streaming that explains the weakness of baseline schemes (e.g., predictive coding, memoryless binning, etc.) and illustrates the advantage of our proposed hybrid scheme over these. Techniques from multi-terminal source coding are used to derive a new lower bound on the compression rate and identify cases when the hybrid coding scheme is close to optimal. We discuss qualitatively the interplay between the parameters of our information theoretic model and the statistical models used in simulations.

We consider the problem of transmitting confidential messages over a two-user broadcast multiple-input multiple-output (MIMO) channel. Surprisingly, the capacity region of this setting under a covariance matrix constraint was shown by Liu et al. to be rectangular. That is, there is no tension, and both users can attain their respective MIMO wiretap capacities, simultaneously. In this work, we provide a new derivation of this result by proposing an alternative achievability scheme for the corner point of the capacity region. This derivation, in addition to being considerably shorter and simpler than the original, also provides a practical transmission scheme, in the sense that the codes used are scalar (single-antenna) ones. We use two main ingredients. The first is the explicit optimal input covariance matrix of Bustin et al. for the MIMO wiretap channel under a covariance matrix constraint, which we also re-derive in a simple manner. The second is a dirty-paper variant of a recently proposed optimal scheme for the MIMO wiretap channel, which uses scalar codes. The proposed treatment demonstrates the connection between the confidential broadcast problem and the MIMO wiretap one: the former almost reduces to the latter, except for the use of dirty-paper coding which is not mandatory in MIMO wiretap; the work sheds light on the reason for this difference.

In this paper, we present nontrivial upper and lower bounds on the secrecy capacity of the degraded Gaussian diamond-wiretap channel and identify several ranges of channel parameters where these bounds coincide with useful intuitions. Furthermore, we investigate the effect of the presence of an eavesdropper on the capacity. We consider the following two scenarios regarding the availability of randomness: 1) a common randomness is available at the source and the two relays and 2) a randomness is available only at the source and there is no available randomness at the relays. We obtain the upper bound by taking into account the correlation between the two relay signals and the availability of randomness at each encoder. For the lower bound, we propose two types of coding schemes: 1) a decode-and-forward scheme where the relays cooperatively transmit the message and the fictitious message and 2) a partial DF scheme incorporated with multicoding in which each relay sends an independent partial message and the whole or partial fictitious message using dependent codewords.

We consider a streaming communication system where the source packets must be played back sequentially at the destination and study the associated average playback delay. We assume that all the source packets are available before the start of transmission at the transmitter and consider the case of an i.i.d. erasure channel with perfect feedback. We first consider the case when the receiver buffer can be arbitrarily large, and show that the average playback delay remains bounded in the length of the stream provided that the channel bandwidth is greater than a critical threshold. Our analysis involves the application of martingale theory to study the transient behaviour of a one dimensional random walk with drift. Conversely when the channel bandwidth is smaller than the above threshold, the average playback delay increases linearly with the stream length. We also consider the finite buffer case and analyse the playback delay of a greedy dynamic bandwidth scheme. We further show through simulations that the achievable delay with a finite receiver buffer is close to the infinite buffer case for moderately large buffer values.

The column Hamming distance of a convolutional code determines the error correction capability when streaming over a class of packet erasure channels. We show that the column sum rank parallels column Hamming distance when streaming over a network with link failures. We prove rank analogues of several column distance properties and introduce a new family of convolutional codes that maximize the column sum rank up to the code memory. Our construction involves finding a class of super-regular matrices that preserve this property after multiplication with non-singular block diagonal matrices in the ground field.

The broadcast channel with independent secret keys is studied. In this scenario, a common message has to be securely broadcast to two legitimate receivers in the presence of an eavesdropper. The transmitter shares with each legitimate receiver an independent secret key of arbitrary rate. These keys can either be used as one-time pads to encrypt the common message or can be interpreted as fictitious messages used as randomization resources for wiretap coding. Both approaches are discussed and the secrecy capacity is derived for various cases. Depending on the qualities of the legitimate and eavesdropper channels, either a one-time pad, wiretap coding, or a combination of both turns out to be capacity-achieving.

We study a source-broadcasting problem involving an erasure broadcast channel with feedback. The receivers each require a certain fraction of a source sequence, and we are interested in the minimum latency, or transmission time, required to serve them all. We first show that for a two-user broadcast channel, a point-to-point outer bound can always be achieved. For broadcasting to three users, we propose a queue-based hybrid digital-analog coding scheme that achieves optimal performance for the duration of analog transmissions. We propose a method of characterizing the number of analog transmissions that can be sent, which involves solving a linear program, and furthermore give sufficient conditions for which all users can be optimal. In some cases, we find that users can be point-to-point optimal regardless of their distortion constraints. Finally, we propose a channel coding phase for when the analog transmissions are insufficient in meeting user demands and provide simulations that highlight the benefits of feedback.

We study the secure degrees of freedom (d.o.f.) of helper-assisted Gaussian wiretap channel with shared key between the transmitter and the helper. Given that the rate of the key scales with power as γ/2 log SNR, we show that secure d.o.f. is min{1+γ / 2, 1}. The achievability proof combines real interference alignment with the artificial noise transmission technique. Using the shared key we sample common artificial noise symbols from a PAM constellation at the transmitter and the helper and transmit them in the null space of the legitimate receiver's channel. We further sample independent noise symbols, also from the same PAM constellation, at the transmitter and the helper and align these symbols at the legitimate receiver. The noise symbols together occupy sufficient dimensions to mask the message at the eavesdropper. A converse proof, which extends the technique of Xie and Ulukus to incorporate common randomness, establishes the optimality of secure d.o.f..We also extend the result to the case with M helpers with two different assumptions on the key sharing structure: the transmitter shares a common key or distinct keys with the M helpers. The exact secure d.o.f.s for these two cases, before they saturate to 1, are proved to be M+γ /M+1 and M+Mγ/M+1 respectively.

The articles in this special issue highlight recent advances along with the remaining challenges in the field of physical-layer communications security.

We establish upper and lower bounds on the secrecy capacity of the degraded Gaussian diamond-wiretap channel, and identify several ranges of channel parameters where these bounds coincide with useful intuitions. Furthermore, we investigate the effect of the presence of an eavesdropper on the capacity. We consider the following two scenarios: 1) common randomness is available at the source and the two relays and 2) randomness is available only at the source, and there is no randomness at the relays. Our upper bounds are established by taking into account the correlation between the two relay signals and the available randomness at the encoders, which generalize the techniques recently developed for the case without secrecy constraint. For the lower bounds, we propose two types of coding schemes: 1) decode-and-forward schemes where the relays cooperatively transmit the message and the fictitious message and 2) partial decode-and-forward schemes incorporated with multicoding in which each relay sends an independent partial message and the whole or partial fictitious message using dependent codewords.

We study low-delay erasure correction codes in a real-time streaming setup. The encoder observes a stream of source packets and outputs the channel packets in a causal fashion, which are broadcast to two receivers over burst-erasure channels. Each receiver must decode the source packets sequentially with a deadline of Ti, while its channel can introduce an erasure burst of maximum length Bi, where i ∈ {1,2} and w.l.o.g. B2> B1. We study the associated capacity as a function of the burst lengths and decoding deadlines. We observe that the operation of the system can be divided into two main regimes. The so-called large-delay regime corresponds to the case when either T1≥ B2or T2≥ B1+ B2. We show that for these parameters, the optimal code is obtained through simple modifications of previously proposed single-user codes by Martinian et al. and the diversity embedded streaming codes proposed by Badr et al. When both T12and T21+ B2, the system is said to be in the low-delay regime. We propose a new code construction and establish its optimality when T2≥ T1+ B1. In the case when T21+ B1, we establish upper and lower bounds on the capacity and characterize the exact capacity when either T1= B1or T2= B2. Our upper bounds in the low-delay regime are based on novel information theoretic arguments that capture the tension between the decoding constraints at the two receivers.

State-dependent parallel networks with a common state-cognitive helper is studied, in which K transmitters wish to send K messages to their corresponding receivers over K state-corrupted parallel channels, and a helper who knows the state information noncausally wishes to assist these receivers to cancel state interference. Furthermore, the helper also has its own message to be sent simultaneously to its corresponding receiver. Since the state information is known only to the helper, but not to other transmitters, transmitter-side state cognition and receiver-side state interference are mismatched. Our focus is on the high state power regime, i.e., the state power goes to infinity. Three (sub)models are studied. Model I serves as a basic model, which consists of only one transmitter-receiver (with state corruption) pair in addition to a helper that assists the receiver to cancel state in addition to transmitting its own message. Model II consists of two transmitter-receiver pairs in addition to a helper, and only one receiver is interfered by a state sequence. Model III generalizes model I to include multiple transmitter- receiver pairs with each receiver corrupted by independent state. For all models, the inner and outer bounds on the capacity region are derived, and comparison of the two bounds yields characterization of either full or partial boundary of the capacity region under various channel parameters.

In this paper, we consider secure broadcasting over fast fading channels. Assuming imperfect main channel state information (CSI) at the transmitter, we first provide an upper and a lower bounds on the ergodic secrecy capacity when a common message is broadcasted to multiple legitimate receivers in the presence of one eavesdropper. For this case, we show that the secrecy rate is limited by the legitimate receiver having, on average, the worst main channel link. Then, we present an expression for the achievable secrecy sum-rate when each legitimate receiver is interested in an independent message. The special cases of high SNR, perfect and no-main CSI are also analyzed. Numerical results are presented to illustrate the obtained results for the case of independent but not necessarily identically distributed Rayleigh fading channels.

The use of secret keys in broadcast channels with receiver side information is studied. The particular scenario is analyzed where a transmitter wants to send two confidential messages to two receivers, while keeping an external eavesdropper ignorant. Each receiver has one of the confidential messages as side information available for decoding. In addition to that, the transmitter shares independent secret keys of arbitrary rates with both receivers. The secret keys can be used in different ways: They can act as one-time pads to encrypt the confidential messages or they can be used as randomization resources for wiretap coding. Both approaches are discussed and an achievable rate region based on superposition coding is established for the one-time pad approach. For the wiretap coding approach, the secrecy capacity for degraded channels is derived. In the optimal coding scheme, the available secret keys are used as the randomization part of the wiretap code to keep the eavesdropper ignorant. In establishing the capacity region, a new upper bound on the sum-rate is derived. This bound shows that in an optimal coding scheme, in the degraded case, the total equivocation-rate of the (opposite) secret-keys at the legitimate receivers must equal the equivocation-rate of the secret-keys at the eavesdropper, when informed about the messages.

The state-dependent parallel Gaussian channel with a common helper is investigated, in which transmitters 1 and 2 transmit two messages respectively to receivers 1 and 2 over the parallel channel. Furthermore, both parallel subchannels can be corrupted by independent state sequences, respectively, which are unknown to both transmitters and receivers. There is a common helper that knows the states noncausally and assists communication between transmitters and receivers. Our focus is on the high state power regime, i.e., the state power goes to infinity. Two Gaussian models are studied with model I having only receiver 1 interfered by the state and with model II having both receivers interfered by independent states. Each model has its unique challenge to address. For both models, inner and outer bounds on the capacity region are derived, and comparison of the two bounds leads to capacity results under certain channel parameters.

The problem of sending a secret message over the multiple-input multiple-output (MIMO) wiretap Gaussian channel is studied. While the capacity of this channel is known, it is not clear how to construct optimal coding schemes that achieve this capacity. In this work we show how to use linear operations along with successive interference cancellation in order to reduce the problem to that of designing optimal codes for the single-antenna additive-noise Gaussian wiretap channel. Much like popular communication techniques in the absence of an eavesdropper, the data is carried over parallel streams. The design approach is flexible enough to allow for using the same scalar wiretap code over all streams, or alternatively to use different scalar wiretap codes over parallel sub-channels without successive interference cancellation. This approach is applicable to more involved secrecy settings, by adjusting the linear operations performed by the encoder, and by jointly processing several channel uses.

We study a successive segmentation-based coding scheme for broadcasting a binary source over a multi-receiver erasure broadcast channel. Each receiver has a certain demand on the fraction of source symbols to be reconstructed, and its channel is a memoryless erasure channel. We study the minimum achievable latency at the source to simultaneously meet all the receiver constraints. We consider a class of schemes that partition the source sequence into multiple segments and apply a systematic erasure code to each segment. We formulate the optimal choice of segment sizes and code-rates in this class of schemes as a linear programming problem and provide an explicit solution.We further show that the optimal solution can be interpreted as a successive segmentation scheme that naturally adjusts when users are added or deleted from the system. Numerical plots indicate significant gains over a baseline separation-based coding scheme.

The state-dependent broadcast channel with a helper is investigated, in which a transmitter wishes to send messages to two receivers via a broadcast channel. The channel is corrupted by an independent and identically distributed (i.i.d.) state sequence which is known to neither the transmitter nor the receivers. A helper that knows the state sequence noncausally assists the broadcast transmission to cancel state interference. Two scenarios are studied. In scenario 1, the transmitter sends one message to both receivers, and in scenario II, the transmitter sends two private messages respectively to two receivers. Our focus is on the Gaussian channel with additive state. Inner and outer bounds are derived for both scenarios. By comparing the inner and outer bounds, capacity/capacity region are characterized under various ranges of channel parameters. Practical impact of the model and results are discussed.

The problem of sending a secret message over the Gaussian multiple-input multiple-output wiretap channel is studied. In a recent work, we have proposed a layered coding scheme where a scalar wiretap code is used in each layer, and successive interference cancellation (SIC) is carried at the legitimate receiver. By a proper rate allocation across the layers, we showed that this scheme satisfies the secrecy constraint at the eavesdropper and achieves the secrecy capacity. However, the existence of the scalar codes was based upon a random coding argument. In this work we take a further step and show how the scheme can be based upon any codes that are good for the ordinary (non-secrecy) additive white Gaussian noise channel. As any stage of the SIC process is equivalent to achieving a corner point of a Gaussian multiple-access channel (MAC) capacity region, the class of codes used needs to be good for the MAC under SIC. Since in the secrecy analysis of our layered scheme, it suffices at each stage to consider a genie-aided eavesdropper that performs SIC, the coding task reduces to guaranteeing secrecy for corner points of induced MACs to the eavesdropper. Structured generation of such codes from ordinary ones is discussed.

We study a multi-antenna broadcast channel with two legitimate receivers and an external eavesdropper. We assume that the channel matrix of the eavesdropper is unknown to the legitimate terminals but satisfies a maximum rank constraint. As our main result we characterize the associated secrecy degrees of freedom for the broadcast channel with common and private messages. We show that a direct extension of the single-user wiretap codebook does not achieve the secrecy degrees of freedom. Our proposed optimal scheme involves decomposing the signal space into a common subspace, which can be observed by both receivers, and private subspaces which can be observed by only one of the receivers, and carefully transmitting a subset of messages in each subspace. We also consider the case when each user's private message must additionally remain confidential from the other legitimate receiver and characterize the s.d.o.f. region in this case.

We study the secrecy capacity of fast fading channels under imperfect main channel (between the transmitter and the legitimate receiver) estimation at the transmitter. Lower and upper bounds on the ergodic secrecy capacity are derived for a class of independent identically distributed (i.i.d.) fading channels. The achievable rate follows from a standard wiretap code in which a simple on-off power control is employed along with a Gaussian input. The upper bound is obtained using an appropriate correlation scheme of the main and eavesdropper channels and is the best known upper bound so far. The upper and lower bounds coincide with recently derived ones in case of perfect main CSI. Furthermore, the upper bound is tight in case of no main CSI, where the secrecy capacity is equal to zero. Asymptotic analysis at high and low signal-to-noise ratio (SNR) is also given. At high SNR, we show that the capacity is bounded by providing upper and lower bounds that depend on the channel estimation error. At low SNR, however, we prove that the secrecy capacity is asymptotically equal to the capacity of the main channel as if there were no secrecy constraint. Numerical results are provided for i.i.d. Rayleigh fading channels.

A setup involving zero-delay sequential transmission of a vector Markov source over a burst erasure channel is studied. A sequence of source vectors is compressed in a causal fashion at the encoder, and the resulting output is transmitted over a burst erasure channel. The destination is required to reconstruct each source vector with zero-delay, but those source sequences that are observed either during the burst erasure, or in the interval of length W following the burst erasure need not be reconstructed. The minimum achievable compression rate is called the rate-recovery function. We assume that each source vector is independent identically distributed (i.i.d.) across the spatial dimension and is sampled from a stationary, first-order Markov process across the temporal dimension. For discrete sources, the case of lossless recovery is considered, and upper and lower bounds on the rate-recovery function are established. Both these bounds can be expressed as the rate for predictive coding, plus a term that decreases at least inversely with the recovery window length W. For Gauss-Markov sources and a quadratic distortion measure, upper and lower bounds on the minimum rate are established when W = 0. These bounds are shown to coincide in the high resolution limit. Finally, another setup involving i.i.d. Gaussian sources is studied and the raterecovery function is completely characterized in this case.

We consider the sequential transmission of a stream of messages over a block-fading multi-input-multi-output channel. A new message arrives at the beginning of each coherence block, and the decoder is required to output each message sequentially, after a delay of T coherence blocks. In the special case when T = 1, the setup reduces to the quasi-static fading channel. We establish the optimal diversity-multiplexing tradeoff (DMT) in the high signal-to-noise-ratio (SNR) regime, and show that it equals T times the DMT of the quasi-static channel. The converse is based on utilizing the delay constraint to amplify a local outage event associated with a message, globally across all the coherence blocks. This approach appears to be new. We propose two coding schemes that achieve the optimal DMT. The first scheme involves interleaving of messages, such that each message is transmitted across T consecutive coherence blocks. This scheme requires the knowledge of the delay constraint at both the encoder and decoder. Our second coding scheme involves a sequential tree code and is delay universal, i.e., the knowledge of the decoding delay is not required by the encoder. However, in this scheme, we require the coherence block length to increase as log (SNR), in order to attain the optimal DMT. Finally, we discuss the case when multiple messages arrive at uniform intervals within each coherence period. Through a simple example, we exhibit the suboptimality of interleaving and propose another scheme that achieves the optimal DMT.

We study broadcasting of two confidential messages to two groups of receivers over independent parallel subchannels. One group consists of an arbitrary number of receivers, interested in a common message, whereas the other group has only one receiver. Each message must be confidential from the receiver(s) in the other group. Each of the subchannels is assumed to be degraded in a certain fashion. While corner points of the capacity region of this setup were characterized in earlier works, we establish the complete capacity region, and show the optimality of a superposition coding technique. For Gaussian channels, we establish the optimality of a Gaussian input distribution by applying an extremal information inequality. By extending our coding scheme to block-fading channels, we demonstrate significant performance gains over a baseline time-sharing scheme.

We study the secret message capacity of an ergodic block fading wiretap channel with partial channel state information at the transmitter and perfect channel state information at the receivers, under both a short term power constraint (STPC) and a long term power constraint (LTPC). We consider that in addition to the statistics of the main and the eavesdropper channel state information (CSI), the sender is provided by the legitimate receiver with a q-bit feedback, at the beginning of each coherence block, through an error-free public channel, with capacity q bits. We establish upper and lower bounds on the secrecy capacity. We show that the lower and the upper bounds coincide asymptotically as q → ∞. When applied to Rayleigh fading channels, we show that, a 4-bit feedback achieves about 90% of the secrecy capacity when perfect main CSI is available at the transmitter. Finally, asymptotic analysis at high and low Signal-to-Noise Ratio (SNR) is presented. It is found that the capacity is bounded at high-SNR, whereas at asymptotically low-SNR, the lower bounds and the upper bound scale linearly with SNR under STPC. Furthermore, subject to LTPC, the capacity at low-SNR is equal to the capacity of the main channel without secrecy constraint and with perfect CSI at both the transmitter and the receiver, under a mild condition on the fading statistics. We also show that a positive secrecy rate is achievable even when the feedback is at the end of each coherence block and q=1.

We study low-delay error correction codes for streaming recovery over a class of packet-erasure channels that introduce both burst-erasures and isolated erasures. We propose a simple, yet effective class of codes whose parameters can be tuned to obtain a tradeoff between the capability to correct burst and isolated erasures. Our construction generalizes previously proposed low-delay codes which are effective only against burst erasures. We establish an information theoretic upper bound on the capability of any code to simultaneously correct burst and isolated erasures and show that our proposed constructions meet the upper bound in some special cases. We discuss the operational significance of column-distance and column-span metrics and establish that the rate 1/2 codes discovered by Martinian and Sundberg [IT Trans. 2004] through a computer search indeed attain the optimal column-distance and column-span tradeoff. Numerical simulations over a Gilbert-Elliott channel model and a Fritchman model show significant performance gains over previously proposed low-delay codes and random linear codes for certain range of channel parameters.

We study near optimal error correction codes for real-time communication. In our setup the encoder must operate on an incoming source stream in a sequential manner, and the decoder must reconstruct each source packet within a fixed playback deadline of T packets. The underlying channel is a packet erasure channel that can introduce both burst and isolated losses. We first consider a class of channels that in any window of length T +1 introduce either a single erasure burst of a given maximum length B, or a certain maximum number N of isolated erasures. We demonstrate that for a fixed rate and delay, there exists a tradeoff between the achievable values of B and N, and propose a family of codes that is near optimal with respect to this tradeoff. We also consider another class of channels that introduce both a burst and an isolated loss in each window of interest and develop the associated streaming codes. All our constructions are based on a layered design and provide significant improvements over baseline codes in simulations over the Gilbert-Elliott channel.

We study sequential streaming of Gauss-Markov sources over a burst-erasure channel. In any sliding window of length L, the channel introduces a single erasure burst of maximum length B. The encoder observes a sequence of vector Gaussian sources, where the vectors are i.i.d. across the spatial dimension and correlated across the temporal dimension. The encoder output can depend on all source vectors observed up to that time but not on any future source vectors. The decoder is required to reconstruct the source vectors instantaneously and within a quadratic distortion constraint of D, except those source vectors that either appear during the erasure periods or a recovery period of W following each erasure burst. We focus on time-invariant encoders and establish upper and lower bounds on the minimum compression rate R(L, B, W, D). Our lower bound is obtained by making connection to a Gaussian multi-terminal source coding problem. The upper bound is based on distributed source coding, but requires a careful analysis of the achievable rate. Numerical comparisons indicate that the proposed technique provides significant gains over other baseline schemes.

We study two multiple-access scenarios with encoders that are informed only intermittently. The first is the Gaussian multiple-access channel with an intermittent feedback link. Here we assume that, depending on the current binary state which evolves in a memoryless fashion, the previous channel output is either revealed to the two encoders or not. For this scenario we obtain an outer bound on the capacity region that approaches the capacity region without feedback when the probability that the channel output will be fed back approaches zero. We also propose an inner bound that converges to the capacity region with ideal feedback and the capacity region with no feedback in the associated extreme cases. In the second scenario the encoders always observe ideal feedback, and in addition they can crib intermittently. For this scenario we establish the capacity region for the special class of semi-deterministic multiple-access channels. The capacity is achieved using the Superposition Block Markov Coding technique of Cover and Leung. For both scenarios the outer bounds are tighter than those obtained by revealing the underlying state sequence non-causally to the encoders.

A state-dependent Gaussian Z-interference channel model is investigated in the regime of high state power, in which transmitters 1 and 2 communicate with receivers 1 and 2, and only receiver 2 is interfered by transmitter 1's signal and a random state sequence. The state sequence is known noncausally only to transmitter 1, not to the corresponding transmitter 2. A layered coding scheme is designed for transmitter 1 to help interference cancelation at receiver 2 (using a cognitive dirty paper coding) and to transmit its own message to receiver 1. Inner and outer bounds are derived, and are further analyzed to characterize the boundary of the capacity region either fully or partially for all Gaussian channel parameters. Our results imply that the capacity region of such a channel with mismatched transmitter-side state cognition and receiver-side state interference is strictly less than that of the corresponding channel without state, which is in contrast to Costa type of dirty channels, for which dirty paper coding achieves the capacity of the corresponding channels without state.

We study a lossy source-broadcasting problem involving the transmission of a binary source over a two-receiver erasure broadcast channel. The motivation of our work stems from the problem faced by a server that wishes to singly broadcast content to a diverse set of users with fractional source reconstruction requirements. In this problem, the server wishes to minimize the overall network latency incurred (measured by the number of channel uses per source symbol) when faced with users of heterogeneous channel qualities, computing capabilities, content demand etc. We provide two complementary approaches to this problem. The first approach is to consider the problem from a joint source-channel coding formulation. Under this formulation, we provide both inner and outer bounds for the network latency under an erasure distortion criterion. Alternatively, the second approach employs rateless coding and formulates an optimization problem so as to find a degree distribution that minimizes the network latency. We compare both approaches with numerical simulations.

A two-transmitter Gaussian multiple access wiretap channel with multiple antennas at each of the nodes is investigated. The channel matrices of the legitimate users are fixed and revealed to all the terminals, whereas the channel matrices of the eavesdropper are arbitrarily varying and only known to the eavesdropper. The secrecy degrees of freedom (s.d.o.f.) region under a strong secrecy constraint is characterized. A transmission scheme that orthogonalizes the transmit signals of the two users at the intended receiver, and uses a single-user wiretap code for each user, is shown to achieve the s.d.o.f. region. The converse involves establishing an upper bound on a weighted-sum-rate expression. This is accomplished by using induction, where at each step one combines the secrecy and multiple-access constraints associated with an adversary eavesdropping a carefully selected group of sub-channels.

We study modulo-sum computation of two binary source sequences over a two-user erasure multiple access channel. The channel is modeled as a binary-input, erasure multiple access channel, which can be in one of three states-either the channel output is a modulo-sum of the two input symbols, or the channel output equals the input symbol on the first link and an erasure on the second link, or vice versa. The associated state sequence is independent and identically distributed. Unlike previously studied multiple-access channels, the proposed channel is not matched to the modulo-sum function and therefore we expect simple cut-set upper bounds to be far from capacity. In this paper, we establish a new upper bound on the modulo-sum capacity that is tighter than the cut-set bound. The key step in establishing this new bound is to provide suitable side information to the encoders to reduce the setup to a compound multiple-access channel and then capture the tension across multiple receivers required to compute the modulo-sum function. In our lower bound, it suffices to use identical linear codebooks at the two encoders. When a (strictly) causal feedback of the channel state is available to the encoders, we present a simple coding scheme that can achieve a rate larger than our upper bound for the case without feedback. This shows that the modulo-sum capacity is increased with feedback. An extension to the case of lossy reconstruction is also treated briefly.

In this paper, we investigate the problem of optimal content cache management for HTTP adaptive bit rate (ABR) streaming over wireless networks. Specifically, in the media cloud, each content is transcoded into a set of media files with diverse playback rates, and appropriate files will be dynamically chosen in response to channel conditions and screen forms. Our design objective is to maximize the quality of experience (QoE) of an individual content for the end users, under a limited storage budget. Deriving a logarithmic QoE model from our experimental results, we formulate the individual content cache management for HTTP ABR streaming over wireless network as a constrained convex optimization problem. We adopt a two-step process to solve the snapshot problem. First, using the Lagrange multiplier method, we obtain the numerical solution of the set of playback rates for a fixed number of cache copies and characterize the optimal solution analytically. Our investigation reveals a fundamental phase change in the optimal solution as the number of cached files increases. Second, we develop three alternative search algorithms to find the optimal number of cached files, and compare their scalability under average and worst complexity metrics. Our numerical results suggest that, under optimal cache schemes, the maximum QoE measurement, i.e., mean-opinion-score (MOS), is a concave function of the allowable storage size. Our cache management can provide high expected QoE with low complexity, shedding light on the design of HTTP ABR streaming services over wireless networks.

We introduce a framework to study fundamental limits of sequential coding of Markov sources under an error propagation constraint. An encoder sequentially compresses a sequence of vector-sources that are spatially i.i.d. but temporally correlated according to a Markov process. The channel erases up to B packets in a single burst, but reveals all other packets to the destination. The destination is required to reproduce all the source-vectors instantaneously and in a loss less manner, except those sequences that occur in a window of length B+W following the start of the erasure burst. We define a rate-recovery function R(B, W), the minimum compression rate that can be achieved in this framework, and develop upper and lower bounds for first-order Markov sources. For the special class of linear diagonally correlated deterministic sources, we propose a new coding technique -- prospicient coding -- that achieves the rate-recovery function. Finally, a lossy extension to the rate-recovery function is also studied for a class of Gaussian sources where the source is temporally and spatially i.i.d. and the decoder aims to recover a collection of past K sources with a quadratic distortion measure. The optimal rate-recovery function is compared with the sub-optimal techniques including forward error correction coding (FEC) and Wyner-Ziv coding, and performance gains are quantified.

In this paper, we investigate the problem of how to cache a set of media files with optimal streaming rates, under HTTP adaptive bit rate streaming over wireless networks. The design objective is to achieve the optimal expected QoE under a limited storage budget, which is measured by the logarithmic relation between the required bit rate and the actual streaming bit rate. We formulate the content cache management of streaming files as a constrained optimization problem. Lagrange multiplier method is employed, and we obtain the numerical solution of the optimal streaming files. Particularly, we characterize the properties of the solution, and find there is a fundamental phase change in the optimal solution as the number of cached files grows. Moreover, the simulation results indicate that with the increase of cache size, more copies of different bit rate should be cached for a better QoE. Our comprehensive investigation reveals insightful guidelines to provide HTTP ABR streaming services over wireless networks.

We study computation of a modulo-sum of two binary source sequences over a two-user erasure multiple access channel. Each sender observes an independent and equiprobable binary sequence and the receiver is interested in computing the modulo-sum of these two sequences. The channel is modelled as a binary-input, erasure multiple access channel, which can be in one of three states - either the channel output is a modulo-sum of the two input symbols, or the channel output equals the input symbol on the first link and an erasure on the second link, or it equals the input symbol on the second link and an erasure on the first link. The associated state sequence is independent and identically distributed. We establish upper and lower bounds on the modulo-sum capacity. Our coding scheme uses either the compute-and-forward or the decode-and-forward techniques. The upper bound is obtained by a genie aided argument that reduces the setup to a compound multiple-access channel. It is in general is tighter than a simple upper bound obtained by revealing one of the messages to the decoders. We also briefly consider the case when a strictly causal state feedback is available to the encoders and establish that such feedback can increase the modulo-sum capacity.

We study private broadcasting of two messages to two groups of users over reversely degraded parallel channels. Group 1 has two users, both interested in a common message whereas group 2 has only one user. The message for each group of users needs to be kept confidential from the other group. We characterize the capacity region for a special degradation structure of the channels and establish the optimality of a superposition codebook where codewords of a secure product-codebook form the cloud centers and codewords of a secure multicast codebook form the satellite codewords. An extension to Gaussian channels with a sum-power constraint is also obtained, where the optimality of Gaussian codebooks is established using an extremal inequality.

We study the secret message capacity of an ergodic block fading wiretap channel with partial channel state information at the transmitter and perfect channel state information at the receivers. We consider that in addition to the statistics of the main and the eavesdropper channel state information (CSI), the sender is provided by the legitimate receiver with a q-bit feedback, at the beginning of each coherence block, through an error-free feedback channel, with capacity q bits. We establish upper and lower bounds on the secrecy capacity. We show that a positive secrecy rate is achievable even when the feedback is at the end of each coherence block and q = 1. We also show that the lower and the upper bounds coincide asymptotically as q → ∞. Finally, asymptotic analysis at high Signal-to-Noise Ratio (SNR) are presented where it is found that the capacity is bounded at high-SNR and present a simple suboptimal scalar quantizer that is capacity achieving, without the need of any numerical optimization, as q → ∞. When applied to Rayleigh fading channels, we show that, at high-SNR, a 4-bit feedback achieves 90% of the secrecy capacity when perfect main CSI is available at the transmitter.

We study the problem of broadcasting a Gaussian source over a Gaussian broadcast channel to two users with individual source-channel bandwidth mismatches, under a quadratic distortion measure. Specifically we study the tradeoff between the achievable distortion pairs between the two users. The case when the bandwidth-expansion factors of the two users are identical has been well studied in the literature and to our best knowledge remains an open problem. Surprisingly, when the bandwidth expansion factors are different, we characterize a range of values where both the users simultaneously attain their point-to-point optimal distortion. Furthermore in the high signal-to-noise ratio regime, this set includes nearly all points where the weaker user has the higher bandwidth expansion factor. In other cases, we propose an achievable tradeoff between the distortion pairs.

We study secret-key agreement over a non-coherent block-fading multiple input multiple output (MIMO) wiretap channel. We give an achievable scheme based on training and source emulation and analyze the rate in the high SNR regime. Based on this analysis we find the optimal number of antennas to use for training. Our main result is that if the sum of the number of antennas at Alice and Bob is larger than the coherence time of the channel, the achievable rate does not depend on the number of antennas at Eve. In this case source emulation is not needed, and using only training is optimal. We also consider the case when there is no public channel available. In this case we show that secret-key agreement is still possible by using the wireless channel for discussion, giving the same number of secure degrees of freedom as in the case with a public channel.

We study the secret-key capacity in a joint source-channel coding setup-the terminals are connected over a discrete memoryless channel and have access to side information, modelled as a pair of discrete memoryless source sequences. As our main result, we establish the upper and lower bounds on the secret-key capacity. In the lower bound expression, the equivocation terms of the source and channel components are functionally additive even though the coding scheme generates a single secret-key by jointly taking into account the source and channel equivocations. Our bounds coincide, thus establishing the capacity, when the underlying wiretap channel can be decomposed into a set of independent, parallel, and reversely degraded channels. For the case of parallel Gaussian channels and jointly Gaussian sources we show that Gaussian codebooks achieve the secret-key capacity. In addition, when the eavesdropper also observes a correlated side information sequence, we establish the secret-key capacity when both the source and channel of the eavesdropper are a degraded version of the legitimate receiver. We finally also treat the case when a public discussion channel is available, propose a separation based coding scheme, and establish its optimality when the channel output symbols of the legitimate receiver and eavesdropper are conditionally independent given the input.

A two-receiver MIMO broadcast-wiretap channel is considered where the channel state of the eavesdropper is arbitrarily varying. It is assumed that the eavesdropper knows this channel state perfectly whereas the legitimate nodes have no knowledge of it. It is further assumed that the eavesdropper experiences no additive noise. The channel between the transmitter and the two legitimate receivers is a constant MIMO Gaussian broadcast channel. This paper establishes the secrecy degrees of freedom region for transmitting a common-confidential message as well as a private- confidential message to each receiver. It is observed that a straightforward extension of single user random binning does not achieve the optimal secrecy degrees of freedom (s.d.o.f.) region. The proposed coding scheme that achieves the s.d.o.f. region involves simultaneous diagonalization of the channel matrices of the two legitimate receivers using the generalized singular value decomposition (GSVD) as well as a particular \emph{structured binning} across codebooks that minimizes the rate of the fictitious message. While the focus is on achieving weak secrecy for ease of exposition, an outline is provided on how the results can be extended for achieving strong secrecy.

Biometric secret-key binding inherently requires signal processing and error correction schemes due to noisy measurement readings. Two previously proposed strategies, Quantization Index Modulation (QIM) and Wyner-Ziv (WZ) coding, are studied in the context of bio metric key binding. We characterize the tradeoff between key rate leakage and key rate-reconstruction distortion, showing that while WZ coding has a better rate-leakage tradeoff than QIM, the latter has a better rate-reconstruction tradeoff. A new strategy is proposed to combine the merits of these schemes. Known as distortion-enhanced Wyner-Ziv coding (DE-WZ), this scheme is demonstrated to exhibit improved flexibility based on numerical results for a uniform source model and scalar quantization.

A rechargeable battery may be used to partially protect the privacy of information contained in a household's electrical load profile. We represent the system as a finite state model to make tractable the computation of the rate of information leakage. Specifically, we use a trellis algorithm to estimate the mutual information rate between the battery's input and output loads. We show that stochastic battery policies can leak 26% less information than a so-called best-effort algorithm (that holds the output load constant whenever possible). We finally describe the extension of the technique to more realistic models of the battery system.

We study delay constrained sequential streaming over block fading channels. The transmitter observes a stream of messages, one message in each coherence block, and the receiver needs to output a sequence of messages, each with a fixed delay of T coherence blocks. We characterize the associated diversity-multiplexing tradeoff (DMT) for this model. The proposed coding scheme involves a semi-infinite random Gaussian tree-code and a sequential decision directed decoder. The converse applies an outage amplification argument that exploits the delay constraint to amplify the error event associated with a single message to an entire sequence of messages.

We study the noncoherent capacity of secret-key agreement with public discussion over independent identically distributed (i.i.d.) Rayleigh fading wireless channels, where neither the sender nor the receivers have access to instantaneous channel state information (CSI). We present two results. At high signal-to-noise ratio (SNR), the secret-key capacity is bounded in SNR, regardless of the number of antennas at each terminal. Second, for a system with a single antenna at both the legitimate and the eavesdropper terminals and an arbitrary number of transmit antennas, the secret-key capacity-achieving input distribution is discrete, with a finite number of mass points. Numerically we observe that at low SNR, the capacity achieving distribution has two mass points with one of them at the origin.

We study the capacity of secret-key agreement over a wiretap channel with state parameters. The transmitter, the legitimate receiver, and the eavesdropper are connected by a discrete memoryless wiretap channel with a memoryless state sequence. The transmitter and the legitimate receiver generate a secret-key that must be concealed from the eavesdropper. We assume that the state sequence is known noncausally to the transmitter and no public discussion channel is available. We derive lower and upper bounds on the secret-key capacity. The lower bound involves a source-channel codebook for constructing a common reconstruction sequence at the legitimate terminals and then mapping this sequence to a secret-key using a secret-key codebook. For the special case of Gaussian channels with additive interference (secret-keys from dirty paper channel) our bounds differ by 0.5 bit/symbol and coincide in the high signal-to-noise-ratio and high interference-to-noise-ratio regimes. In another special case-symmetric channel state information (CSI)-when the legitimate receiver is also revealed the state sequence, we establish optimality of our lower bound. In addition, only causal side information at the transmitter and the receiver suffices to attain the secret-key capacity in the case of symmetric CSI.

We study a wiretap channel model where the sender has$M $transmit antennas and there are two groups consisting of$J_{1}$and$J_{2}$receivers respectively. Each receiver has a single antenna. We consider two scenarios. First we consider the compound wiretap model — group 1 constitutes the set of legitimate receivers, all interested in a common message, whereas group 2 is the set of eavesdroppers. We establish new lower and upper bounds on the secure degrees of freedom (d.o.f.). Our lower bound is based on the recently proposed real interference alignment scheme. The upper bound provides the first known example which illustrates that the pairwise upper bound used in earlier works is not tight. The second scenario we study is the compound private broadcast channel. Each group is interested in a message that must be protected from the other group. Upper and lower bounds on the d.o.f. are developed by extending the results on the compound wiretap channel.

In Costa's dirty-paper channel, Gaussian random binning is able to eliminate the effect of interference which is known at the transmitter, and thus achieve capacity. We examine a generalization of the dirty-paper problem to a multiple access channel (MAC) setup, where structured (lattice-based) binning seems to be necessary to achieve capacity. In the dirty-MAC, two additive interference signals are present, one known to each transmitter but none to the receiver. The achievable rates using Costa's Gaussian binning vanish if both interference signals are strong. In contrast, it is shown that lattice-strategies (“lattice precoding”) can achieve positive rates, independent of the interference power. Furthermore, in some cases-which depend on the noise variance and power constraints-high-dimensional lattice strategies are in fact optimal. In particular, they are optimal in the limit of high SNR-where the capacity region of the dirty MAC with strong interference approaches that of a clean MAC whose power is governed by the minimum of the users' powers rather than their sum. The rate gap at high SNR between lattice-strategies and optimum (rather than Gaussian) random binning is conjectured to be1/2log2(πe/6) ≈ 0.254 bit. Thus, the doubly dirty MAC is another instance of a network setting, like the Körner-Marton problem, where (linear) structured coding is potentially better than random binning.

Streaming erasure codes encode a source stream to guarantee that each source packet is recovered within a fixed delay at the receiver over a burst-erasure channel. This paper introduces diversity embedded streaming erasure codes (DE-SCo), that provide a flexible trade-off between the channel quality and receiver delay. When the channel conditions are good, the source stream is recovered with a low delay, whereas when the channel conditions are poor the source stream is still recovered, albeit with a larger delay. Information theoretic analysis of the underlying burst-erasure broadcast channel reveals that DE-SCo achieve the minimum possible delay for the weaker user, without sacrificing the performance of the stronger user. A larger class of multicast streaming erasure codes (MU-SCo) that achieve optimal tradeoff between rate, delay and erasure-burst length is also constructed.

We study a non-multicast secure network coding problem with two receivers. First we study a linear-deterministic channel model with two receivers and a collection of eavesdroppers, which generalizes the Ozarow-Wyner wiretap channel II. The secrecy capacity region for independent and common messages is characterized and is achieved by concatenating a coset-coding scheme based on maximum rank distance codes with a repetition code. By applying our coding scheme at the source node of a network that uses an underlying generic network code we also establish the secrecy capacity region of a network coding problem with two sinks and one sender node.

The role of multiple antennas for secure communication is investigated within the framework of Wyner's wiretap channel. We characterize the secrecy capacity in terms of generalized eigenvalues when the sender and eavesdropper have multiple antennas, the intended receiver has a single antenna, and the channel matrices are fixed and known to all the terminals, and show that a beamforming strategy is capacity-achieving. In addition, we study a masked beamforming scheme that radiates power isotropically in all directions and show that it attains near-optimal performance in the high SNR regime. Insights into the scaling behavior of the capacity in the large antenna regime as well as extensions to ergodic fading channels are also provided.

The capacity of the Gaussian wiretap channel model is analyzed when there are multiple antennas at the sender, intended receiver and eavesdropper. The associated channel matrices are fixed and known to all the terminals. A computable characterization of the secrecy capacity is established as the saddle point solution to a minimax problem. The converse is based on a Sato-type argument used in other broadcast settings, and the coding theorem is based on Gaussian wiretap codebooks.

We study secret-key agreement protocols over a wiretap channel controlled by a state parameter. The secret-key capacity is established when the wiretap channel is discrete and memoryless, the sender and receiver are both revealed the underlying state parameter, and no public discussion is allowed. An optimal coding scheme involves a two step approach — (i) design a wiretap codebook assuming that the state parameter is also known to the eavesdropper (ii) generate an additional secret key by exploiting the uncertainty of the state parameter at the eavesdropper. When unlimited public discussion is allowed between the legitimate terminals, we provide an upper bound on the secret-key capacity and establish its tightness when the channel outputs of the legitimate receiver and eavesdropper satisfy a conditional independence property. Numerical results for an on-off fading model suggest that the proposed coding schemes significantly outperform naive schemes that either disregard the contribution of the common state sequence or the contribution of the underlying channel.

We study a multicast extension of streaming burst-erasure codes previously proposed for the single user setting. There are two receivers each interested in the common stream. Each receiver's channel however has a different burst parameter and likewise each receiver tolerates a different delay; both receivers are interested in a common stream. We develop two upper bounding approaches on the streaming multicast capacity. The first upper bound is developed by introducing an erasure channel that introduces periodic erasure bursts, which can corrected due to the multicast property. The second upper bound is based on information theoretic inequalities and is tight at the minimum delay point. Finally we propose a simple multicast code construction by combining the parity checks of two single-user codes.

A joint-source-channel setup for secret-key generation between remote terminals is considered. The sender communicates to the receiver over a discrete memoryless wiretap channel and the sender and receiver observe a pair of correlated discrete memoryless sources. Lower and upper bounds for the secret-key rate are presented and shown to coincide for the case when the underlying channel is a reversely degraded parallel channel. Our setup also provides an operational significance to the rate-equivocation tradeoff of the wiretap channel, and this is illustrated in detail for the Gaussian case.

We study a problem of broadcasting confidential messages to multiple receivers under an information-theoretic secrecy constraint. Two scenarios are considered: 1) all receivers are to obtain a common message; and 2) each receiver is to obtain an independent message. Moreover, two models are considered: parallel channels and fast-fading channels. For the case of reversely degraded parallel channels, one eavesdropper, and an arbitrary number of legitimate receivers, we determine the secrecy capacity for transmitting a common message, and the secrecy sum-capacity for transmitting independent messages. For the case of fast-fading channels, we assume that the channel state information of the legitimate receivers is known to all the terminals, while that of the eavesdropper is known only to itself. We show that, using a suitable binning strategy, a common message can be reliably and securely transmitted at a rate independent of the number of receivers. We also show that a simple opportunistic transmission strategy is optimal for the reliable and secure transmission of independent messages in the limit of large number of receivers.

Practical cooperative diversity protocols often rely on low-cost radios that treat multiple in-band signals as noise and thus require strictly orthogonal transmissions. We analyze the performance of a class of opportunistic relaying protocols that employ simple packet level feedback and strictly orthogonal transmissions. It is shown that the diversity-multiplexing tradeoff of the proposed protocols either matches or outperforms the multi-input-single-output (MISO), zero-feedback performance. These gains indicate that low complexity radios and feedback could be an appealing architecture for future user cooperation protocols.

We describe a method to encode fingerprint biometrics securely for use, e.g., in encryption or access control. The system is secure because the stored data does not suffice to recreate the original fingerprint biometric. Therefore, a breach in database security does not lead to the loss of biometric data. At the same time the stored data suffices to validate a probe fingerprint. Our approach is based on the use of distributed source coding techniques implemented with graph-based codes. We present a statistical model of the relationship between the enrollment biometric and the (noisy) biometric measurement taking during authentication. We describe how to validate or reject a candidate biometric probe given the probe and the stored encoded data. We report the effectiveness of our method as tested on a database consisting of 579 data sets, each containing roughly 15 measurements of a single finger. We thereby demonstrate a working secure biometric system for fingerprints.

Wyner's wiretap channel is generalized to the case when the sender, the receiver and the eavesdropper have multiple antennas. We consider two cases: the deterministic case and the fading case. In the deterministic case, the channel matrices of the intended receiver and the eavesdropper are fixed and known to all the nodes. In the fading case, the channel matrices experience block fading and the sender has only the intended receiver's channel state information (CSI) and statistical knowledge of the eavesdropper's channel. For the deterministic case, a scheme based on the generalized-singular-value-decomposition (GSVD) of the channel matrices is proposed and shown to achieve the secrecy capacity in the high signal-to-noise-ratio (SNR) limit. When the intended receiver has only one antenna (MISO case) the secrecy-capacity is characterized for any SNR. Next, a suboptimal "artificial noise" based scheme is considered. Its performance is characterized and observed to be nearly optimal in the high SNR regime for the MISO case. This scheme extends naturally to the fading case and results are reported for the MISO case. For the independent Rayleigh fading distribution as we simultaneously increase the number of antennas at the sender and the eavesdropper, the secrecy capacity approaches zero if and only if the ratio of the number of eavesdropper antennas to transmitter antennas is at least two.

We consider a generalization of the Gaussian dirty- paper problem to a multiple access setup. There are two additive interferences, one known to each transmitter but none to the receiver. The rates achievable using random binning schemes (i.e. schemes based on Costa's auxiliary random variables) vanish in the limit when the interferences are strong. In contrast, we show that lattice strategies ("lattice preceding") can achieve positive rates independent of the interferences. Furthermore, we derive an outer bound for the capacity region for arbitrary interferences, which is strictly smaller than the clean MAC capacity region. We then show that lattice strategies meet this outer bound for some combinations of noise variance and power constraints. In particular, lattice strategies are optimal in the limit of high SNR. Thus, the dirty MAC is another instance of a network setup, like the Korner-Marton modulo-two sum problem, where linear coding is better than random binning. We also derive lattice transmission schemes and conditions for optimality for the asymmetric case, where there is only one interference which is known to one of the users, and in particular for the helper problem, where the user which knows the interference does not have a message it wishes to transmit.

A generalization of the problem of writing on dirty paper is considered in which one transmitter sends a common message to multiple receivers. Each receiver experiences on its link an additive interference (in addition to the additive noise), which is known noncausally to the transmitter but not to any of the receivers. Applications range from wireless multiple-antenna multicasting to robust dirty paper coding. We develop results for memoryless channels in Gaussian and binary special cases. In most cases, we observe that the availability of side information at the transmitter increases capacity relative to systems without such side information, and that the lack of side information at the receivers decreases capacity relative to systems with such side information. For the noiseless binary case, we establish the capacity when there are two receivers. When there are many receivers, we show that the transmitter side information provides a vanishingly small benefit. When the interference is large and independent across the users, we show that time sharing is optimal. For the Gaussian case, we present a coding scheme and establish its optimality in the high signal-to-interference-plus-noise limit when there are two receivers. When the interference power is large and independent across all the receivers, we show that time-sharing is again optimal. Connections to the problem of robust dirty paper coding are also discussed

We use the distortion side information (DSI) framework to study the gains in information embedding when the encoder exploits sensitivity of the source samples. Our study for the Gaussian source model extends the dirty paper coding result by Costa to the case of a weighted power constraint with the weights only known to the transmitter. A coding scheme based on fixed codebook variable-partition codes is presented for this problem. We also present another coding scheme that exploits the knowledge of DSI and is robust against intentional attacks. Finally, we study a related problem of Wyner-Ziv coding with reliability side information (RSI) at the decoder. This latter setup illustrates that fixed codebook variable-partition codes could also be fundamental in systems that rely on conventional distortion measures

We study the information theoretic limits of communication over asynchronous discrete memoryless channels. The transmitter starts sending a block codeword of length N at a time v uniformly distributed within the interval [1, 2, ..., L]. We assume that the receiver knows L but not v. We give a scaling law of L with respect to N for which reliable communication can be achieved. Specifically, we propose a communication scheme with the property that, unless the asynchrony level L grows at least as eNC, where C denotes the capacity of the synchronized channel, arbitrary low error probability can be achieved. If L grows sub-exponentially in N, the capacity is the same as that of the ordinary synchronized channel. Further, we provide a lower bound to the error probability given a certain channel, codebook, and asynchrony level. This bound together with our scheme shows that, in certain cases, the condition L les eNC(1-delta)for any delta > 0 is an asymptotic necessary and sufficient condition for reliable communication. Finally we extend our analysis to a simple scenario where communication is carried over a Gaussian channel with antipodal signaling +radicP and -radicP. We show that a necessary condition on the amount of power needed in order to guarantee reliable communication is that P must scale as 1/NlogL when L rarr infin

We study the diversity-multiplexing tradeoff in cooperative diversity systems involving multiple relays. We focus on low complexity architectures that do not require simultaneous transmissions on the same frequency band and therefore are amenable to practical implementation with low-cost radios. We show that smart relay selection protocols achieve the same performance as previously proposed protocols that rely on multi-terminal space-time coding. Our study includes both analog and digital relays under a variety of relay selection criteria, and considers the availibility of decision feedback in the network. Our results present an alternative to distributed space-time codes for realizing the potential gains in multiple relay cooperative systems and open new avenues for fruitful interaction between routing and cooperative diversity.

We describe the architecture for our distributed video coding (DVC) system. Some key differences between our work and previous systems include a new method of enabling decoder motion compensation, and the use of serially concatenated accumulate syndrome codes for distributed source coding. To evaluate performance, we compare our system to the H.263+ and H.264/AVC video codecs. Experiments show that our system is comparable to DVC systems from Stanford and Berkeley in the sense that our system performs better than H.263+Intra, but worse than H.263+Inter and H.264/AVC

Cooperative diversity has been recently proposed as a way to form virtual antenna arrays that provide dramatic gains in slow fading wireless environments. However, most of the proposed solutions require distributed space-time coding algorithms, the careful design of which is left for future investigation if there is more than one cooperative relay. We propose a novel scheme that alleviates these problems and provides diversity gains on the order of the number of relays in the network. Our scheme first selects the best relay from a set of M available relays and then uses this "best" relay for cooperation between the source and the destination. We develop and analyze a distributed method to select the best relay that requires no topology information and is based on local measurements of the instantaneous channel conditions. This method also requires no explicit communication among the relays. The success (or failure) to select the best available path depends on the statistics of the wireless channel, and a methodology to evaluate performance for any kind of wireless channel statistics, is provided. Information theoretic analysis of outage probability shows that our scheme achieves the same diversity-multiplexing tradeoff as achieved by more complex protocols, where coordination and distributed space-time coding for M relay nodes is required, such as those proposed by Laneman and Wornell (2003). The simplicity of the technique allows for immediate implementation in existing radio hardware and its adoption could provide for improved flexibility, reliability, and efficiency in future 4G wireless systems.

A framework is developed for analyzing capacity gains from user cooperation in slow-fading wireless networks when the number of nodes (network size) is large. The framework is illustrated for the case of a simple multipath-rich Rayleigh-fading channel model. Both unicasting (one source and one destination) and multicasting (one source and several destinations) scenarios are considered. We introduce a meaningful notion of Shannon capacity for such systems, evaluate this capacity as a function of signal-to-noise ratio (SNR), and develop a simple two-phase cooperative network protocol that achieves it. We observe that the resulting capacity is the same for both unicasting and multicasting, but show that the network size required to achieve any target error probability is smaller for unicasting than for multicasting. Finally, we introduce the notion of a network "scaling exponent" to quantify the rate of decay of error probability with network size as a function of the targeted fraction of the capacity. This exponent provides additional insights to system designers by enabling a finer grain comparison of candidate cooperative transmission protocols in even moderately sized networks.

We study potential gains in the spectral efficiency of multi-cell uplink CDMA systems that accrue from assigning users in different parts of a cell to different frequency bands. We develop a criterion that enables a fair comparison with a conventional (non-partitioned) system. Two design scenarios are considered: (i) each user requires a fixed data rate; (ii) each user has a fixed SNR. We show that in both scenarios there is a gain in spectral efficiency - regardless of the particular partitioning strategy employed f an MMSE receiver is used. There is no gain if a decorrelator or a matched filter receiver is used. We validate our results through numerical simulations and provide an intuitive explanation based on the coding-spreading tradeoff.

We study the problem of sending a common message to several users on a channel with side information. Specifically, each user experiences an additive interference which is known only to the sender. The sender has to simultaneously adapt its transmitted signal to all the interferences. We derive upper and lower bounds for the special case of binary channels and derive some optimality conditions.

We describe a practical method for distributed compression of q-ary sources using multi-level serially concatenated-accumulate codes. Our approach works well at high compression rates, and allows for graceful and incremental rate-adaptivity. Simulations show that the compression efficiency is near the information-theoretic limits for correlations between sources that obey a Gaussian or Laplacian distribution.