Coded polynomial aggregation (CPA) enables the master to directly recover a weighted aggregation of polynomial evaluations without individually decoding each term, thereby reducing the number of required worker responses. In this paper, we extend CPA to straggler-aware distributed computing systems and introduce a straggler-aware CPA framework with pre-specified non-straggler patterns, where exact recovery is required only for a given collection of admissible non-straggler sets. Our main result shows that exact recovery of the desired aggregation is achievable with fewer worker responses than required by polynomial coded computing based on individual decoding, and that feasibility is fundamentally characterized by the intersection structure of the non-straggler patterns. In particular, we establish necessary and sufficient conditions for exact recovery in straggler-aware CPA and identify an intersection-size threshold that is sufficient to guarantee exact recovery. We further prove that this threshold becomes both necessary and sufficient when the number of admissible non-straggler sets is sufficiently large. We also provide an explicit construction of feasible CPA schemes whenever the intersection size exceeds the derived threshold. Finally, simulations reveal a sharp feasibility transition at the predicted threshold, providing empirical evidence that the bound is tight in practice.

We characterise bounds on the optimal broadcast rate for a few classes of pliable-index-coding instances. Unlike the majority of currently solved instances, which belong to a special class where all receivers with a certain side-information cardinality are either present or absent, we consider more general instances without this constraint. We devise a novel algorithm that constructs a decoding chain by iteratively adding a message that can be decoded by a receiver whose side information is already in the chain. If the decoding chain cannot proceed due to the absence of a receiver with the required messages, weskipa message by adding it to the chain regardless. We prove that a lower bound on the optimal broadcast rate is a function of the number of skipped messages, across all possible decoding choices of the receivers and any realisation of the algorithm for each decoding choice. While this result is not computationally feasible in isolation, it serves as a basis for deriving explicit lower bounds on the broadcast rate for specific classes of pliable-index-coding instances. These lower bounds depend on the number of absent receivers or the pattern of their side-information sets. Specifically, we explicitly characterise the optimal broadcast rate for instances with up to and including four absent receivers with any side-information pattern, as well as instances where the side-information sets are nested in particular ways.

Coded elastic computing, introduced by Yang et al. in 2018, is a technique designed to mitigate the impact of elasticity in cloud computing systems, where machines can be preempted or be added during computing rounds. This approach utilizes maximum distance separable (MDS) coding for both storage and download in matrix-matrix multiplications. The proposed scheme is unable to tolerate stragglers and has high encoding complexity and upload cost. In 2023, we addressed these limitations by employing uncoded storage and Lagrange-coded download. However, it results in a large storage size. To address the challenges of storage size and upload cost, in this paper, we focus on Lagrange-coded elastic computing based on uncoded download. We propose a new class of elastic computing schemes, using Lagrange-coded storage with uncoded download (LCSUD). Our proposed schemes address both elasticity and straggler challenges while achieving lower storage size, reduced encoding complexity, and upload cost compared to existing methods.

Coded elastic computing enables virtual machines to be preempted for high-priority tasks while allowing new virtual machines to join ongoing computation seamlessly. This paper addresses coded elastic computing for matrix-matrix multiplications with straggler tolerance by encoding both storage and download using Lagrange codes. In 2018, Yang et al. introduced the first coded elastic computing scheme for matrix-matrix multiplications, achieving a lower computational load requirement. However, this scheme lacks straggler tolerance and suffers from high upload cost. Zhong et al. (2023) later tackled these shortcomings by employing uncoded storage and Lagrange-coded download. However, their approach requires each machine to store the entire dataset. This paper introduces a new class of elastic computing schemes that utilize Lagrange codes to encode both storage and download, achieving a reduced storage size. The proposed schemes efficiently mitigate both elasticity and straggler effects, with a storage size reduced to a fraction 1/L of Zhong et al.'s approach, at the expense of doubling the download cost. Moreover, we evaluate the proposed schemes on AWS EC2 by measuring computation time under two different tasks allocations: heterogeneous and cyclic assignments. Both assignments minimize computation redundancy of the system while distributing varying computation loads across machines.

We propose a power-controlled differentially private decentralized learning algorithm designed for a set of clients aiming to collaboratively train a common learning model. The network is characterized by a row-stochastic adjacency matrix, which reflects different channel gains between the clients. In our privacy-preserving approach, both the transmit power for model updates and the level of injected Gaussian noise are jointly controlled to satisfy a given privacy and energy budget. We show that our proposed algorithm achieves a convergence rate of$O(\log T)$, where$T$is the horizon bound in the regret function. Furthermore, our numerical results confirm that our proposed algorithm outperforms existing works.

This paper introduces and analyzes a search and retrieval model that adopts key semantic communication principles from retrieval-augmented generation. We specifically present an information-theoretic analysis of a remote document retrieval system operating over a symbol erasure channel. The proposed model encodes the feature vector of a query, derived from term-frequency weights of a language corpus by using a repetition code with an adaptive rate dependent on the contextual importance of the terms. At the decoder, we select between two documents based on the contextual closeness of the recovered query. By leveraging a jointly Gaussian approximation for both the true and reconstructed similarity scores, we derive an explicit expression for the retrieval error probability, i.e., the probability under which the less similar document is selected. Numerical simulations on synthetic and real-world data (Google NQ) confirm the validity of the analysis. They further demonstrate that assigning greater redundancy to critical features effectively reduces the error rate, highlighting the effectiveness of semantic-aware feature encoding in error-prone communication settings.

We study a privacy-preserving data-sharing setting where a privatizer transforms private data into a sanitized version observed by an authorized reconstructor and two unauthorized adversaries, each with access to side information correlated with the private data.The reconstructor is evaluated under a distortion function, while each adversary is evaluated using a separate loss function. The privatizer ensures the reconstructor distortion remains below a fixed threshold while maximizing the minimum loss across the two adversaries. This two-adversary setting models cases where individual users cannot reconstruct the data accurately, but their combined side information enables estimation within the distortion threshold. The privatizer maximizes individual loss while permitting accurate reconstruction only through collaboration. This echoes secret-sharing principles, but with lossy rather than perfect recovery. We frame this as a constrained data-driven minimax optimization problem and propose a data-driven training procedure that alternately updates the privatizer, reconstructor, and adversaries. We also analyze the Gaussian and binary cases as special scenarios where optimal solutions can be obtained. These theoretical optimal results are benchmarks for evaluating the proposed minimax training approach.

We consider the problem of communication-constrained collaborative personalized mean estimation under a privacy constraint in an environment of several agents continuously receiving data according to arbitrary unknown agent-specific distributions. A consensus-based algorithm is studied under the framework of differential privacy in order to protect each agent’s data. We give a theoretical convergence analysis of the proposed consensus-based algorithm for any bounded unknown distributions on the agents’ data, showing that collaboration provides faster convergence than a fully local approach where agents do not share data, under an oracle decision rule and under some restrictions on the privacy level and the agents’ connectivity, which illustrates the benefit of private collaboration in an online setting under a communication restriction on the agents. The theoretical faster-than-local convergence guarantee is backed up by several numerical results.

Distributed computations, such as distributed matrix multiplication, can be vulnerable to significant security issues, notably Byzantine attacks. These attacks may target either worker nodes or servers, potentially leading to faulty results that can significantly degrade the overall performance. Therefore, detecting Byzantine attackers and mitigating their effects are crucial in such systems. Motivated by the goal of establishing a secure decentralized matrix-multiplication system, we first introduce a verification method named Common Tag, inspired by the well-known Freivalds’ algorithm, able to verify the multiplication results independent of their associated input matrices. Then, we propose two schemes for sparse matrix multiplication where a group of nodes collaboratively performs a computation task over a logical ring. We consider a subset of Byzantine nodes in the system that may arbitrarily corrupt either their result or any other result passing through them. In Scheme I considering the highly sparse nature of input matrices, we assume that each node has sufficient capacity to store the entire input matrices, and the nodes forward the read-only versions of their computed blocks so that other nodes cannot corrupt them. In Scheme II, we relax the above assumptions, firstly, by considering a limited storage capacity for each node. Secondly, we introduce more powerful adversaries capable of corrupting other nodes’ results by relaxing the read-only assumption. The results demonstrate the feasibility of both schemes and show a significant improvement in terms of distortion over the case where no detection happens. The results also provide a trade-off between the computational complexity required at each node and the reconstruction distortion in both schemes.

We consider the problem of collaborative personalized mean estimation under a privacy constraint in an environment of several agents continuously receiving data according to arbitrary unknown agent-specific distributions. In particular, we provide a method based on hypothesis testing coupled with differential privacy and data variance estimation. Two differential privacy mechanisms protecting the releases of each agent’s current sample mean and two data variance estimation schemes are proposed, and we provide a theoretical convergence analysis of the proposed algorithm for any bounded unknown distributions on the agents’ data, showing that collaboration provides faster convergence than a fully local approach where agents do not share data. Moreover, we provide analytical performance curves for the case with an oracle class estimator, i.e., the class structure of the agents, where agents receiving data from distributions with the same mean are considered to be in the same class, is known. The theoreticalfaster-than-localconvergence guarantee is backed up by extensive numerical results showing that for a considered scenario with 200 agents from two or three classes the proposed approach indeed converges much faster than a fully local approach, and performs comparably to the ideal (all-data-public) case. This illustrates the benefit of private collaboration in an online setting.

ConsiderLusers, who each hold private data, and one fusion center who must compute a function of the private data of theLusers. To accomplish this task, each user may utilize a public and noiseless broadcast channel in a non-interactive manner. In this setting, and in the absence of any additional resources such as secure links, we study the optimal communication rates and minimum information leakages on the private user data that are achievable. Specifically, we study the information leakage of the user data at the fusion center (beyond the knowledge of the function output), as well as at predefined groups of colluding users who eavesdrop one another. We derive the capacity region when the user data is independent, and inner and outer regions for the capacity region when the user data is correlated.

In 2018, Yang et al. introduced a novel and effective approach, using maximum distance separable (MDS) codes, to mitigate the impact of elasticity in cloud computing systems. This approach is referred to as coded elastic computing. Some limitations of this approach include that it assumes all virtual machines have the same computing speeds and storage capacities, and it cannot tolerate stragglers for matrix-matrix multiplications. In order to resolve these limitations, in this paper, we introduce a new combinatorial optimization framework, named uncoded storage coded transmission elastic computing (USCTEC), for heterogeneous speeds and storage constraints, aiming to minimize the expected computation time for matrix-matrix multiplications, under the consideration of straggler tolerance. Within this framework, we propose optimal solutions with straggler tolerance under relaxed storage constraints. Moreover, we propose a heuristic algorithm that considers heterogeneous storage constraints. Our results demonstrate that the proposed algorithm outperforms baseline solutions utilizing cyclic storage placements, in terms of both expected computation time and storage size.

We introduce the paradigm of validated decentralized learning for undirected networks with heterogeneous data and possible adversarial infiltration. We require ($a$) convergence to a global empirical loss minimizer when adversaries are absent, and$(\boldsymbol{b})$either detection of adversarial presence or convergence to an admissible consensus model in their presence. This contrasts sharply with the traditional byzantine-robustness requirement of convergence to an admissible consensus irrespective of the adversarial configuration. To this end, we propose the Valid protocol which, to the best of our knowledge, is the first to achieve a validated learning guarantee. Moreover, Valid offers an$O(1/T)$convergence rate (under pertinent regularity assumptions), and computational and communication complexities comparable to non-adversarial distributed stochastic gradient descent. Remarkably, Valid retains optimal performance metrics in adversary-free environments, sidestepping the robustness penalties observed in prior byzantine-robust methods. A distinctive aspect of our study is a heterogeneity metric based on the norms of individual agents' gradients computed at the global empirical loss minimizer. This not only provides a natural statistic for detecting significant byzantine disruptions but also allows us to prove the optimality of Valid in wide generality. Lastly, our numerical results reveal that, in the absence of adversaries, Validcon-verges faster than state-of-the-art byzantine robust algorithms, while when adversaries are present, Valid terminates with each honest agent either converging to an admissible consensus or declaring adversarial presence in the network.

Consider a scenario involving multiple users and a fusion center. Each user possesses a sequence of bits and can communicate with the fusion center through a one-way public channel. The fusion center's task is to compute the sum of all the sequences under the privacy requirement that a set of colluding users, along with the fusion center, cannot gain more than a predetermined amount$\delta$of information, measured through mutual information, about the sequences of other users. Our first contribution is to characterize the minimum amount of necessary communication between the users and the fusion center, as well as the minimum amount of necessary shared randomness at the users. Our second contribution is to establish a connection between secure summation and secret sharing by showing that secret sharing is necessary to generate the local randomness needed for private summation, and prove that it holds true for any$\delta\geqslant 0$.

Consider the problem of storing data in a distributed manner over T servers. Specifically, the data needs to (i) be recoverable from any$\tau $servers, and (ii) remain private from any z colluding servers, where privacy is quantified in terms of mutual information between the data and all the information available at any z colluding servers. For this model, our main results are (i) the fundamental trade-off between storage size and the level of desired privacy, and (ii) the optimal amount of local randomness necessary at the encoder. As a byproduct, our results provide an optimal lower bound on the individual share size of ramp secret sharing schemes under a more general leakage symmetry condition than the ones previously considered in the literature.

In this paper, we propose a sparse matrix multiplication in a decentralized setting, where a set of worker nodes wishes to compute a task collaboratively over a logical ring. We consider a subset of Byzantine nodes in the system who want to maliciously corrupt the result by corrupting their own computed blocks. In particular, the main focus of this paper is to compute the result with the least possible distortion by identifying the Byzantine nodes and re-assigning their tasks to the benign nodes. Our results demonstrate the feasibility of our proposed decentralized scheme and provide a trade-off between the computational complexity required at each worker node and the reconstruction distortion.

In cloud computing systems, elastic events and stragglers increase the uncertainty of the system, leading to computation delays. Coded elastic computing (CEC) introduced by Yang et al. in 2018 is a framework which mitigates the impact of elastic events using Maximum Distance Separable (MDS) coded storage. It proposed a CEC scheme for both matrix-vector multiplication and general matrix-matrix multiplication applications. However, in these applications, the proposed CEC scheme cannot tolerate stragglers due to the limitations imposed by MDS codes. In this paper we propose a new elastic computing scheme using uncoded storage and Lagrange coded computing approaches. The proposed scheme can effectively mitigate the effects of both elasticity and stragglers. Moreover, it produces a lower complexity and smaller recovery threshold compared to existing coded storage based schemes.

Consider the problem of storing data in a distributed manner over T servers. Specifically, the data needs to (i) be recoverable from any τ servers, and (ii) remain private from any z colluding servers, where privacy is quantified in terms of mutual information between the data and all the information available at any z colluding servers. For this model and under a leakage symmetry requirement at the servers, our main results are (i) the fundamental trade-off between storage size and the level of desired privacy, and (ii) the optimal amount of local randomness necessary at the encoder. As a byproduct, our results provide an optimal lower bound on the individual share size of ramp secret sharing schemes under a more general leakage symmetry condition than the ones previously considered in the literature.

We consider the problem of collaborative personalized mean estimation under a privacy constraint in an environment of several agents continuously receiving data according to arbitrary unknown agent-specific distributions. In particular, we provide a method based on hypothesis testing coupled with differential privacy. Two privacy mechanisms are proposed and we provide a theoretical convergence analysis of the proposed algorithm for any bounded unknown distributions on the agents’ data. Numerical results show that for a considered scenario the proposed approach converges much faster than a fully local approach where agents do not share data, and performs comparably to ideal performance where all data is public. This illustrates the benefit of private collaboration in an online setting.

In this work we propose RELDEC, a novel approach for sequential decoding of moderate length low-density parity-check (LDPC) codes. The main idea behind RELDEC is that an optimized decoding policy is subsequently obtained via reinforcement learning based on a Markov decision process (MDP). In contrast to our previous work, where an agent learns to schedule only a single check node (CN) within a group (cluster) of CNs per iteration, in this work we train the agent to schedule all CNs in a cluster, and all clusters in every iteration. That is, in each learning step of RELDEC an agent learns to schedule CN clusters sequentially depending on a reward associated with the outcome of scheduling a particular cluster. We also modify the state space representation of the MDP, enabling RELDEC to be suitable for larger block length LDPC codes than those studied in our previous work. Furthermore, to address decoding under varying channel conditions, we propose agile meta-RELDEC (AM-RELDEC) that employs meta-reinforcement learning. The proposed RELDEC scheme significantly outperforms standard flooding and random sequential decoding for a variety of LDPC codes, including codes designed for 5G new radio.

This work establishes that the physical layer can be used to perform information-theoretic authentication in additive white Gaussian noise (AWGN) channels, as long as the adversary is not omniscient. The model considered consists of an encoder, decoder, and adversary, where the adversary knows the message given to the encoder, has a non-causal noisy observation of the encoder’s transmission and may use unlimited transmission power, while the decoder observes a noisy version of the sum of the encoder and adversary’s outputs. A method to modify a generic existing channel code to enable authentication is presented. This method relies on injecting message-dependent noise into the transmission and accepting the transmission as authentic only if the correct noise levels for the decoded message are observed. One drawback to this method is that the encoder must still transmit a low-power signal in the case where there is no message to send. It is shown that this modification costs an asymptotically negligible amount of the coding rate, while still enabling authentication as long as the adversary’s observation is not noiseless. Also notable is that this modification is not (asymptotically) a function of the statistical characterization of the adversary’s channel and no secret key is required. We believe these features will pave the way for a robust practical implementation. Using these results, the channel-authenticated capacity is calculated and shown to be equal to the non-adversarial channel capacity. As our results will show, information-theoretic authentication in AWGN channels is possible without the need for the legitimate party to have a model-based advantage over the adversary. While this modular scheme is designed for use in the given channel model, it is applicable to a wide range of settings.

Consider L users, who each holds private data, and one fusion center who must compute a function of the private data of the L users. To accomplish this task, each user can make a single use of a public and noiseless broadcast channel. In this setting, and in the absence of any additional resources such as secure links, we study the optimal communication rates and minimum information leakages on the private user data that are achievable. Specifically, we study the information leakage of the user data at the fusion center (beyond the knowledge of the function output), as well as at predefined groups of colluding users who eavesdrop one another. We derive the capacity region when the user data is independent, and inner and outer regions for the capacity region when the user data is correlated.

Information leakage to a guessing adversary in index coding is studied, where some messages in the system are sensitive and others are not. The non-sensitive messages can be used by the server like secret keys to mitigate leakage of the sensitive messages to the adversary. We construct a deterministic linear coding scheme, developed from the rank minimization method based on fitting matrices (Bar-Yossef et al. 2011). The linear scheme leads to a novel upper bound on the optimal information leakage rate, which is proved to be tight over all deterministic scalar linear codes. We also derive a converse result from a graph-theoretic perspective, which holds in general over all deterministic and stochastic coding schemes.

We formulate a new variant of the private information retrieval (PIR) problem where the user is pliable, i.e., interested in any message from a desired subset of the available dataset, denoted as pliable private information retrieval (PPIR). We consider the setup where a dataset consisting of f messages is replicated in n noncolluding databases and classified into Γ classes. For this setup, the user wishes to retrieve any λ ≥ 1 messages from multiple desired classes, while revealing no information about the identity of the desired classes to the databases. We term this problem multi-message PPIR (M-PPIR) and introduce the single-message PPIR (PPIR) problem as an elementary special case of M-PPIR. We first derive converse bounds on the M-PPIR download rate, followed by achievable schemes. As a result, we show that the PPIR capacity for f messages and Γ classes matches the PIR capacity with n noncolluding databases and Γ messages. Thus, enabling flexibility, i.e., pliability, where privacy is only guaranteed for classes, but not for messages as in classical PIR, allows to trade-off privacy versus download rate. A similar insight is shown to hold for the general case of M-PPIR.

We consider straggler resiliency in decentralized learning using stochastic gradient descent under the notion of network differential privacy (DP). In particular, we extend the recently proposed framework of privacy amplification by decentralization by Cyffers and Bellet to include training latency—comprising both computation and communication latency. Analytical results on both the convergence speed and the DP level are derived for training over a logical ring for both a skipping scheme (which ignores the stragglers after a timeout) and a baseline scheme that waits for each node to finish before the training continues. Our results show a trade-off between training latency, accuracy, and privacy, parameterized by the timeout of the skipping scheme. Finally, results when training a logistic regression model on a real-world dataset are presented.

Private computation in a distributed storage system (DSS) is a generalization of the private information retrieval (PIR) problem. In such a setting, a user wishes to compute a function of$f$messages stored in$n$noncolluding coded databases, i.e., databases storing data encoded with an$[n,k]$linear storage code, while revealing no information about the desired function to the databases. We consider the problem of private linear computation (PLC) for coded databases. In PLC, a user wishes to compute a linear combination over the$f$messages while keeping the coefficients of the desired linear combination hidden from the databases. For a DSS setup where data is stored using a code from a particular family of linear storage codes, we derive an outer bound on the PLC rate, which is defined as the ratio of the desired amount of information and the total amount of downloaded information. In particular, the proposed converse is valid for any number of messages and linear combinations, and depends on the rank of the coefficient matrix obtained from all linear combinations. Further, we present a PLC scheme with rate equal to the outer bound and hence settle the PLC capacity for the considered class of linear storage codes. Interestingly, the PLC capacity matches the maximum distance separable coded capacity of PIR for the considered class of linear storage codes.

Private information retrieval protocols guarantee that a user canprivatelyandlosslesslyretrieve a single file from a database stored across multiple servers. In this work, we propose to simultaneously relax the conditions of perfect retrievability and privacy in order to obtain improved download rates when all files are stored uncoded on a single server. Information leakage is measured in terms of the average success probability for the server of correctly guessing the identity of the desired file. The main findings are: i) The derivation of the optimal tradeoff between download rate, distortion, and information leakage when the file size isinfinite. Closed-form expressions of the optimal tradeoff for the special cases of “no-leakage” and “no-privacy” are also given. ii) A novel approach based on linear programming (LP) to construct schemes for a finite file size and an arbitrary number of files. The proposed LP approach can be leveraged to find provably optimal schemes with corresponding closed-form expressions for the rate-distortion-leakage tradeoff when the database contains at most four bits. Finally, for a database that contains 320 bits, we compare two construction methods based on the LP approach with a nonconstructive scheme downloading subsets of files using a finite-length lossy compressor based on random coding.

Consider the identification (ID) via channels problem, where a receiver decides whether the transmitted identifier is its identifier, rather than decoding it. This model allows to transmit identifiers whose size scales doubly-exponentially in the blocklength, unlike common transmission codes with exponential scaling. Binary constant-weight codes (CWCs) suffice to achieve the ID capacity. Relating parameters of a binary CWC to the minimum distance of a code and using higher-order correlation moments, two upper bounds on binary CWC sizes are proposed. These bounds are also upper bounds on identifier sizes for ID codes constructed by using binary CWCs. We propose two constructions based on optical orthogonal codes (OOCs), which are used in optical multiple access schemes, have constant-weight codewords, and satisfy cyclic cross-correlation and auto-correlation constraints. These constructions are modified and concatenated with outer Reed-Solomon codes to propose new binary CWCs being optimal for ID. Improvements to the finite-parameter performance are shown by using outer codes with larger minimum distance vs. blocklength ratios. We illustrate ID regimes for which our ID code constructions perform significantly better than existing constructions.

Two main privacy models for sanitising datasets are differential privacy (DP) and syntactic privacy. The former restricts individual values’ impact on the output based on the dataset while the latter restructures the dataset before publication to link any record to multiple sensitive data values. Besides both providing mechanisms to sanitise data, these models are often applied independently of each other and very little is known regarding how they relate. Knowing how privacy models are related can help us develop a deeper understanding of privacy and can inform how a single privacy mechanism can fulfil multiple privacy models. In this paper, we introduce a framework that determines if the privacy mechanisms of one privacy model can also guarantee privacy for another privacy model. We apply our framework to understand the relationship between DP and a form of syntactic privacy called t-closeness. We demonstrate, for the first time, how DP and t-closeness can be interpreted in terms of each other by introducing generalisations and extensions of both models to explain the transition from one model to the other. Finally, we show how applying one mechanism to guarantee multiple privacy models increases data utility compared to applying separate mechanisms for each privacy model.

We consider the problem of private polynomial computation (PPC) from a distributed storage system (DSS). In such setting a user wishes to compute a multivariate polynomial of degree at most$g$over$f$variables (or messages) stored in$n$noncolluding coded databases, i.e., databases storing data encoded with an$[n,k]$linear storage code, while revealing no information about the desired polynomial evaluation to the databases. For a DSS setup where data is stored using linear storage codes, we derive an outer bound on the PPC rate, which is defined as the ratio of the (minimum) desired amount of information and the total amount of downloaded information, and construct two novel PPC schemes. In the first scheme, we consider Reed-Solomon coded databases with Lagrange encoding, which leverages ideas from recently proposed star-product private information retrieval and Lagrange coded computation. The second scheme considers the special case of coded databases with systematic Lagrange encoding. Both schemes yield improved rates, while asymptotically, as$f\rightarrow \infty $, the systematic scheme gives a significantly better computation retrieval rate compared to all known schemes up to some storage code rate that depends on the maximum degree of the candidate polynomials.

We propose to extend the concept of private information retrieval by allowing for distortion in the retrieval process and relaxing the perfect privacy requirement at the same time. In particular, we study the tradeoff between download rate, distortion, and user privacy leakage, and show that in the limit of large file sizes this tradeoff can be captured via a novel information-theoretical formulation for datasets with a known distribution. Moreover, for scenarios where the statistics of the dataset is unknown, we propose a new deep learning framework by leveraging a generative adversarial network approach, which allows the user to learn efficient schemes from the data itself, minimizing the download cost. We evaluate the performance of the scheme on a synthetic Gaussian dataset as well as on the MNIST, CIFAR-10, and LSUN datasets. For the MNIST, CIFAR-10, and LSUN datasets, the data-driven approach significantly outperforms a nonlearning-based scheme which combines source coding with multiple file download.

Our extensive real measurements over Amazon EC2 show that the virtual instances often have different computing speeds even if they share the same configurations. This motivates us to study heterogeneous Coded Storage Elastic Computing (CSEC) systems where machines, with different computing speeds, join and leave the network arbitrarily over different computing steps. In CSEC systems, a Maximum Distance Separable (MDS) code is used for coded storage such that the file placement does not have to be re-defined with each elastic event. Computation assignment algorithms are used to minimize the computation time given computation speeds of different machines. While previous studies of heterogeneous CSEC do not include stragglers - the slow machines during the computation, we develop a new framework in heterogeneous CSEC that introduces straggler tolerance. Based on this framework, we design a novel algorithm using our previously proposed approach for heterogeneous CSEC such that the system can handle any subset of stragglers of a specified size while minimizing the computation time. Furthermore, we establish a trade-off in computation time and straggler tolerance. Another major limitation of existing CSEC designs is the lack of practical evaluations using real applications. In this paper, we evaluate the performance of our designs on Amazon EC2 for applications of the power iteration and linear regression. Evaluation results show that the proposed heterogeneous CSEC algorithms outperform the state-of-the-art designs by more than 30%.

Consider the identification (ID) via channels problem, where a receiver wants to decide whether the transmitted identifier is its identifier, rather than decoding the identifier. This model allows to transmit identifiers whose size scales doubly-exponentially in the blocklength, unlike common transmission (or channel) codes whose size scales exponentially. It suffices to use binary constant-weight codes (CWCs) to achieve the ID capacity. By relating the parameters of a binary CWC to the minimum distance of a code and using higher-order correlation moments, two upper bounds on the binary CWC size are proposed. These bounds are shown to be upper bounds also on the identifier sizes for ID codes constructed by using binary CWCs. We propose two code constructions based on optical orthogonal codes, which are used in optical multiple access schemes, have constant-weight codewords, and satisfy cyclic cross-correlation and autocorrelation constraints. These constructions are modified and concatenated with outer Reed-Solomon codes to propose new binary CWCs optimal for ID. Improvements to the finite-parameter performance of both our and existing code constructions are shown by using outer codes with larger minimum distance vs. blocklength ratios. We also illustrate ID performance regimes for which our ID code constructions perform significantly better than existing constructions.

We study the information leakage to a guessing adversary in zero-error source coding. The source coding problem is defined by a confusion graph capturing the distinguishability between source symbols. The information leakage is measured by the ratio of the adversary's successful guessing probability after and before eavesdropping the codeword, maximized over all possible source distributions. Such measurement under the basic adversarial model where the adversary makes a single guess and the guess is regarded successful if and only if the estimator sequence equals to the true source sequence is known as the maximum min-entropy leakage or the maximal leakage in the literature. We develop a single-letter characterization of the optimal normalized leakage under the basic adversarial model, together with an optimum-achieving memoryless stochastic mapping scheme. An interesting observation is that the optimal normalized leakage is equal to the optimal compression rate with fixed-length source codes, both of which can be simultaneously achieved by some deterministic coding schemes. We then extend the leakage measurement to generalized adversarial models where the adversary makes multiple guesses and allows a certain level of distortion, for which we derive single-letter lower and upper bounds.

Private information retrieval protocols guarantee that a user can privately and losslessly retrieve a single file from a database stored across multiple servers. In this work, we propose to simultaneously relax the conditions of perfect retrievability and privacy in order to obtain improved download rates in the single server scenario, i.e., all files are stored uncoded on a single server. In particular, we derive the optimal tradeoff between download rate, distortion, and information leakage when the file size is infinite and the information leakage is measured in terms of the average success probability for the server of correctly guessing the identity of the requested file. Moreover, we present a novel approach based on linear programming to construct schemes for a finite file size and an arbitrary number of files. When the database contains at most four bits, this approach can be leveraged to find provably optimal schemes.

We study the information leakage to a guessing adversary in index coding with a general message distribution. Under both vanishing-error and zero-error decoding assumptions, we develop lower and upper bounds on the optimal leakage rate, which are based on the broadcast rate of the subproblem induced by the set of messages the adversary tries to guess. When the messages are independent and uniformly distributed, the lower and upper bounds match, establishing an equivalence between the two rates.

Linear nested codes, where two or more sub-codes are nested in a global code, have been proposed as candidates for reliable multi-terminal communication. In this article, we consider nested array-based spatially coupled low-density parity-check (SC-LDPC) codes and propose a line-counting based optimization scheme for minimizing the number of dominant absorbing sets in order to improve its performance in the high signal-to-noise ratio regime. Since the parity-check matrices of different nested sub-codes partially overlap, the optimization of one nested sub-code imposes constraints on the optimization of the other sub-codes. To tackle these constraints, a multi-step optimization process is applied first to one of the nested codes, then sequential optimization of the remaining nested codes is carried out based on the constraints imposed by the previously optimized sub-codes. Results show that the order of optimization has a significant impact on the number of dominant absorbing sets in the Tanner graph of the code, resulting in a trade-off between the performance of a nested code structure and its optimization sequence: the code which is optimized without constraints has fewer harmful structures than the code which is optimized with constraints. We also show that for certain code parameters, dominant absorbing sets in the Tanner graphs of all nested codes are completely removed using our proposed optimization strategy.

We study the problem of strong coordination of the actions of two nodes X and Y that communicate over a discrete memoryless channel (DMC) such that the actions follow a prescribed joint probability distribution. We propose two novel random coding schemes and a polar coding scheme for this noisy strong coordination problem, and derive inner and outer bounds for the respective strong coordination capacity region. The first scheme is a joint coordination-channel encoding scheme that utilizes the randomness provided by the communication channel to reduce the amount of local randomness required to generate the sequence of actions at Node Y. Based on this random coding scheme, we provide a characterization of the capacity region for a special case of the noisy strong coordination setup, namely, when the DMC is a deterministic channel. The second scheme exploits separate coordination and channel encoding where local randomness is extracted from the channel after decoding. Moreover, by leveraging the random coding results for this problem, we present an example in which the proposed joint encoding scheme is able to strictly outperform the separate encoding scheme in terms of achievable communication rate for the same amount of injected randomness into both systems. Thus, we establish the sub-optimality of the separation of strong coordination and channel encoding with respect to the communication rate over the DMC in this problem. Finally, the third scheme is a joint coordination-channel polar coding scheme for strong coordination. We show that polar codes are able to achieve the established inner bound to the strong noisy coordination capacity region and thus provide a constructive alternative to a random coding proof. Our polar coding scheme also offers a constructive solution to a channel simulation problem where a DMC and shared randomness are employed together to simulate another DMC.

In unsecured communications settings, ascertaining the trustworthiness of received information, called authentication, is paramount. We consider keyless authentication over an arbitrarily-varying channel, where channel states are chosen by a malicious adversary with access to noisy versions of transmitted sequences. We have shown previously that a channel condition termed U-overwritability is a sufficient condition for zero authentication capacity over such a channel, and also that with a deterministic encoder, a sufficiently clear-eyed adversary is essentially omniscient. In this paper, we show that even if the authentication capacity with a deterministic encoder and an essentially omniscient adversary is zero, allowing a stochastic encoder can result in a positive authentication capacity. Furthermore, the authentication capacity with a stochastic encoder can be equal to the no-adversary capacity of the underlying channel in this case. We illustrate this for a binary channel model, which provides insight into the more general case.

We consider the problem of storing data in a distributed manner over T servers. We require the data (i) to be recoverable from the T servers, and (ii) to remain private from any T -1 colluding servers, where privacy is quantified in terms of mutual information between the data and all the information available at the T -1 colluding servers. For this model, we determine (i) the fundamental trade-off between storage size and the level of desired privacy, (ii) the optimal amount of local randomness necessary at the encoder, and (iii) an explicit low-complexity coding scheme that solely relies on XOR operations and that asymptotically (with the data size) matches the fundamental limits found.

The multi-armed bandit (MAB) problem refers to the dilemma encountered by a gambler when deciding which arm of a multi-armed slot machine to pull in order to maximize the total reward earned in a sequence of pulls. In this paper, we model the scheduling of a node-wise sequential LDPC decoder as a Markov decision process, where the underlying Tanner graph is viewed as a slot machine with multiple arms corresponding to the check nodes. A fictitious gambler decides which check node to pull (schedule) next by observing a reward associated with each pull. This interaction enables the gambler to discover an optimized scheduling policy that aims to reach a codeword output by propagating the fewest possible messages. Based on this policy, we contrive a novel MAB-based node-wise scheduling (MABNS) algorithm to perform sequential decoding of LDPC codes. Simulation results show that the MAB-NS scheme, aided by an appropriate scheduling policy, outperforms traditional scheduling schemes in terms of complexity and bit error probability.

The uplink of a Cloud Radio Access Network (C-RAN) architecture is studied, where decoding in the cloud takes place at distributed decoding processors. To mitigate the impact of straggling decoders in the cloud, the cloud re-encodes the received frames via a linear code before distributing them to the decoding processors, which estimate linear combinations of the codewords. Focusing on Gaussian channels, and assuming the use of lattice codes at the users, we derive the computational rates and frame error probabilities at the cloud. The approach differs from Compute-and-Forward in that the combination of codewords is not caused by the channel but purposefully created in the cloud by encoding the received signals to reduce the decoding delay.

We show in this work that reinforcement learning can be successfully applied to decoding short to moderate length sparse graph-based channel codes. Specifically, we focus on low-density parity check (LDPC) codes, which for example have been standardized in the context of 5G cellular communication systems due to their excellent error correcting performance. These codes are typically decoded via belief propagation iterative decoding on the corresponding bipartite (Tanner) graph of the code via flooding, i.e., all check and variable nodes in the Tanner graph are updated at once. In contrast, in this paper we utilize a sequential update policy which selects the optimum check node (CN) scheduling in order to improve decoding performance. In particular, we model the CN update process as a multi-armed bandit process with dependent arms and employ a Q-learning scheme for optimizing the CN scheduling policy. In order to reduce the learning complexity, we propose a novel graph-induced CN clustering approach to partition the state space in such a way that dependencies between clusters are minimized. Our results show that compared to other decoding approaches from the literature, the proposed reinforcement learning scheme not only significantly improves the decoding performance, but also reduces the decoding complexity dramatically once the scheduling policy is learned.

Large matrix multiplications are central to large-scale machine learning applications. These operations are often carried out on a distributed computing platform with a master server and multiple workers in the cloud operating in parallel. For such distributed platforms, it has been recently shown that coding over the input data matrices can reduce the computational delay, yielding a trade-off between recovery threshold, i.e., the number of workers required to recover the matrix product, and communication load, i.e., the total amount of data to be downloaded from the workers. In this paper, in addition to exact recovery requirements, we impose security and privacy constraints on the data matrices, and study the recovery threshold as a function of the communication load. We first assume that both matrices contain private information and that workers can collude to eavesdrop on the content of these data matrices. For this problem, we introduce a novel class of secure codes, referred to as secure generalized PolyDot (SGPD) codes, that generalize state-of-the-art non-secure codes for matrix multiplication. SGPD codes allow a flexible trade-off between recovery threshold and communication load for a fixed maximum number of colluding workers while providing perfect secrecy for the two data matrices. We then study a connection between secure matrix multiplication and private information retrieval. We specifically assume that one of the data matrices is taken from a public set known to all the workers. In this setup, the identity of the matrix of interest should be kept private from the workers. For this model, we present a variant of generalized PolyDot codes that can guarantee both secrecy of one matrix and privacy for the identity of the other matrix for the case of no colluding servers.

Tensor operations, such as matrix multiplication, are central to large-scale machine learning applications. These operations can be carried out on a distributed computing platform with a master server at the user side and multiple workers in the cloud operating in parallel. For distributed platforms, it has been recently shown that coding over the input data matrices can reduce the computational delay, yielding a tradeoff between recovery threshold and communication load. In this work, we impose an additional security constraint on the data matrices and assume that workers can collude to eavesdrop on the content of these data matrices. Specifically, we introduce a novel class of secure codes, referred to as secure generalized PolyDot codes, that generalizes previously published non-secure versions of these codes for matrix multiplication. These codes extend the state-of-the-art by allowing a flexible trade-off between recovery threshold and communication load for a fixed maximum number of colluding workers.

The joint design of input constellation and low-density parity-check (LDPC) codes to approach the symmetric capacity of the two-user Gaussian multiple access channel is studied. More specifically, multilevel coding is employed at each user to construct a high-order input constellation and the constellations of the users are jointly designed so as to maximize the multiuser shaping gain. At the receiver, each layer of the multilevel coding is jointly decoded among users, while successive cancellation is employed across layers. The LDPC code employed by each user in each layer is designed using EXIT charts to support joint decoding among users for the prescribed per-layer rate and SNR. Numerical simulations are provided to validate the proposed constellation and LDPC code designs.

Authentication in the presence of a malicious adversary consists of either recovering the legitimate transmission or declaring that the adversary has interfered with the transmission. In this work, we present a structured coding scheme for keyless authentication over a discrete memoryless binary-input, symmetric adversarial channel. Our scheme allows for coding rates up to the non-adversarial capacity of the underlying channel, as well as bounded-complexity decoding.

We consider private polynomial computation (PPC) over noncolluding coded databases. In such a setting a user wishes to compute a multivariate polynomial of degree at most g over f variables (or messages) stored in multiple databases while revealing no information about the desired polynomial to the databases. We construct two novel PPC schemes, where the first is a generalization of our previous work in private linear computation for coded databases. In this scheme we consider Reed-Solomon coded databases with Lagrange encoding, which leverages ideas from recently proposed star-product private information retrieval and Lagrange coded computation. The second scheme considers the special case of coded databases with systematic Lagrange encoding. Both schemes yield improved rates compared to the best known schemes from the literature for a small number of messages, while in the asymptotic case the rates match.

We characterise the optimal broadcast rate for a few classes of pliable-index-coding problems. This is achieved by devising new lower bounds that utilise the set of absent receivers to construct decoding chains with skipped messages. This work complements existing works by considering problems that are not complete-S, i.e., problems considered in this work do not require that all receivers with a certain side-information cardinality to be either present or absent from the problem. We show that for a certain class, the set of receivers is critical in the sense that adding any receiver strictly increases the broadcast rate.

Linear nested codes, where two or more subcodes are nested in a global code, have been proposed as candidates for reliable multi-terminal communication. In this paper, we consider nested array-based spatially coupled LDPC codes and propose a line-counting based optimization scheme for minimizing the number of dominant absorbing sets in order to improve its performance in the high signal-to-noise ratio regime. The presented multi-step optimization process is applied first to one of the nested codes, then an optimization of the remaining nested codes is carried out based on these code constraints. We also show that for certain code parameters, dominant absorbing sets in the Tanner graphs of all nested codes can be completely removed using our proposed optimization strategy.

We consider the problem of private computation (PC) in a distributed storage system. In such a setting a user wishes to compute a function of f messages replicated across n noncolluding databases, while revealing no information about the desired function to the databases. We provide an information-theoretically accurate achievable PC rate, which is the ratio of the smallest desired amount of information and the total amount of downloaded information, for the scenario of nonlinear computation. For a large message size the rate equals the PC capacity, i.e., the maximum achievable PC rate, when the candidate functions are the f independent messages and one arbitrary nonlinear function of these. When the number of messages grows, the PC rate approaches an outer bound on the PC capacity. As a special case, we consider private monomial computation (PMC) and numerically compare the achievable PMC rate to the outer bound for a finite number of messages.

The uplink of a cloud radio access network architecture is studied in which decoding at the cloud takes place via network function virtualization on commercial off-the-shelf servers. In order to mitigate the impact of straggling decoders in this platform, a novel coding strategy is proposed, whereby the cloud re-encodes the received frames via a linear code before distributing them to the decoding processors. Transmission of a single frame is considered first, and upper bounds on the resulting frame unavailability probability as a function of the decoding latency are derived by assuming a binary symmetric channel for uplink communications. Then, the analysis is extended to account for random frame arrival times. In this case, the tradeoff between an average decoding latency and the frame error rate is studied for two different queuing policies, whereby the servers carry out per-frame decoding or continuous decoding, respectively. Numerical examples demonstrate that the bounds are useful tools for code design and that coding is instrumental in obtaining a desirable compromise between decoding latency and reliability.

This paper explores the relationship between two ideas in the network information theory: edge removal and strong converses. Edge removal properties state that if an edge of small capacity is removed from a network, the capacity region does not change too much. Strong converses state that, for rates outside the capacity region, the probability of error converges to 1 as the blocklength goes to infinity. Various notions of edge removal and strong converse are defined, depending on how edge capacity and error probability scale with blocklength, and relations between them are proved. Each class of strong converse implies a specific class of edge removal. The opposite directions are proved for deterministic networks. Furthermore, a technique based on a novel, causal version of the blowing-up lemma is used to prove that for discrete memoryless networks, the weak edge removal property-that the capacity region changes continuously as the capacity of an edge vanishes-is equivalent to the exponentially strong converse-that outside the capacity region, the probability of error goes to 1 exponentially fast. This result is used to prove exponentially strong converses for several examples, including the discrete two-user interference channel with strong interference, with only a small variation from traditional weak converse proofs.

The uplink of a cloud radio access network architecture is studied in which decoding at the cloud takes place via network function virtualization (NFV) on commercial off-the-shelf (COTS) servers. In order to mitigate the impact of straggling decoders in the cloud computing platform, a novel coding strategy is proposed, whereby the cloud re-encodes the received frames via a linear code before distributing them to the decoding processors. Upper bounds on the resulting frame unavailability probability (FUP) as a function of the decoding latency are derived by assuming a binary symmetric channel for uplink communications. The bounds leverage large deviation results for correlated variables, and depend on the properties of both the uplink linear channel code adopted at the user and the NFV linear code applied at the cloud. Numerical examples demonstrate that the bounds are useful tools for code design, and that coding is instrumental in obtaining a desirable tradeoff between FUP and decoding latency.

Finite blocklength and second-order (dispersion) results are presented for the arbitrarily-varying channel (AVC), a classical model wherein an adversary can transmit arbitrary signals into the channel. A novel finite blocklength achievability bound is presented, roughly analogous to the random coding union bound for non-adversarial channels. This finite blocklength bound, along with a known converse bound, is used to derive bounds on the dispersion of discrete memoryless AVCs without shared randomness, and with cost constraints on the input and the state. These bounds are tight for many channels of interest, including the binary symmetric AVC. However, the bounds are not tight if the deterministic and random code capacities differ.

We study the problem of private function retrieval (PFR) in a distributed storage system. In PFR the user wishes to retrieve a linear combination of M messages stored in non-colluding (N, K) MDS coded databases while revealing no information about the coefficients of the intended linear combination to any of the individual databases. We present an achievable scheme for MDS coded PFR with a rate that matches the capacity for coded private information retrieval derived recently, R = (1+Rc+Rc2+...+RcM-1)-1=[(1-Rc)/(1-RcM)], where Rc=[K/N] is the rate of the MDS code.

A linear code equivalence between index coding and network coding was shown by El Rouayheb et al., which establishes that for any index-coding instance, there exists a network-coding instance for which any index code can be mapped to a suitable network code, and vice versa. Similarly, for any network-coding instance, there exists an index-coding instance for which a similar code equivalence can be constructed. Effros et al. extended the equivalence to include non-linear codes. Subsequently, we extended the code equivalence to the secure communication setting in the presence of an eavesdropper, in which we impose perfect decodability and secrecy. In this paper, we generalise the equivalence between secure index coding and secure network coding to include non-zero decoding error and non-zero leakage.

Recently, Kumar, Li, and EI Gamal proposed a notion of common information using a variation of a setup used to define Wyner common information rate. This notion, known as the exact common information, is the minimum common randomness required for the exact and separate generation of a pair of correlated discrete memoryless sources. While exact common information rate is not known to have a single-letter characterization, it was shown to equal the Wyner common information rate for the symmetric binary erasure source in Kumar-Li-EI Gamal-ISIT2014. The authors extended this result to establish the equality of the two notions of common information for general noisy typewriter, Z - and erasure sources in Vellambi - Kliewer - Allerton 2016. In this work, we investigate the connection between exact and Wyner common information rates to derive two new implicit conditions (on the joint source distribution) that ensure the equality of the two notions.

We consider binary spatially coupled (SC) low density measurement matrices for low complexity reconstruction of sparse signals via the interval passing algorithm (IPA). The IPA is known to fail due to the presence of harmful sub-structures in the Tanner graph of a binary sparse measurement matrix, so called termatiko sets. In this work we construct array-based (AB) SC sparse measurement matrices via algebraic lifts of graphs, such that the number of termatiko sets in the Tanner graph is minimized. To this end, we show for the column-weight-three case that the most critical termatiko sets can be removed by eliminating all length-12 cycles associated with the Tanner graph, via algebraic lifting. As a consequence, IPA-based reconstruction with SC measurement matrices is able to provide an almost error free reconstruction for significantly denser signal vectors compared to uncoupled AB LDPC measurement matrices.

Keyless authentication is considered in an adversarial point-to-point channel. Namely, a legitimate transmitter and receiver aim to communicate over a noisy channel that may or may not also contain an active adversary, capable of transmitting an arbitrary signal into the channel. If the adversary is not present, then the receiver must successfully decode the message with high probability; if it is present, then the receiver must either decode the message or detect the adversary's presence. Thus, whenever the receiver decodes, it can be certain that the decoded message is authentic. The exact authentication capacity is characterized for discrete-memoryless adversary channels, where the adversary is assumed to know the code but not the message. The authentication capacity is shown to be either zero or equal to the no-adversary capacity, depending on whether the channel satisfies a condition termed overwritability.

It has been shown that a class of spatially coupled low-density generator-matrix (SC-LDGM) code ensembles displays distortion saturation for the lossy binary symmetric source coding problem with the belief propagation guided decimation (BPGD) algorithm, i.e., the BPGD distortion approaches the optimal expected distortion of the underlying ensemble asymptotically in code length. We investigate the distortion performance of a practical class of protograph-based SC-LDGM code ensembles and demonstrate distortion saturation numerically. Moreover, taking advantage of the convolutional structure of the SC-LDGM codes, we propose an efficient windowed encoding (WE) algorithm with two decimation techniques for lowering the WE complexity that maintain distortion performance close to the rate-distortion bound.

We design polar codes for empirical coordination and strong coordination in two-node networks. Our constructions hinge on the fact that polar codes enable explicit low-complexity schemes for soft covering. We leverage this property to propose explicit and low-complexity coding schemes that achieve the capacity regions of both empirical coordination and strong coordination for sequences of actions taking value in an alphabet of prime cardinality. Our results improve previously known polar coding schemes, which (i) were restricted to uniform distributions and to actions obtained via binary symmetric channels for strong coordination, (ii) required a non-negligible amount of common randomness for empirical coordination, and (iii) assumed that the simulation of discrete memoryless channels could be perfectly implemented. As a by-product of our results, we obtain a polar coding scheme that achieves channel resolvability for an arbitrary discrete memoryless channel whose input alphabet has prime cardinality.

This paper reduces multiple-unicast network coding to single-unicast secure network coding and single-unicast network error correction. Specifically, we present reductions that map an arbitrary multiple-unicast network coding instance to a unicast secure network coding instance in which at most one link is eavesdropped, or a unicast network error correction instance in which at most one link is erroneous, such that a rate tuple is achievable in the multiple-unicast network coding instance if and only if a corresponding rate is achievable in the unicast secure network coding instance, or in the unicast network error correction instance. Conversely, we show that an arbitrary unicast secure network coding instance in which at most one link is eavesdropped can be reduced back to a multiple-unicast network coding instance. In addition, we show that the capacity of a unicast network error correction instance in general is not (exactly) achievable.

We analyze the problem of strong coordination over a multi-hop line network in which the node initiating the coordination is a terminal network node. We assume that each node has access to a certain amount of randomness that is local to the node, and that the nodes also have shared common randomness, which are used together with explicit hop-by-hop communication to achieve information-theoretic strong coordination. We derive the trade-offs among the required rates of communication on the network links, the rates of local randomness available at network nodes, and the rate of common randomness to realize strong coordination. We present an achievable coding scheme built using multiple layers of channel resolvability codes, and establish several settings in which this scheme offers the best possible trade-offs among network resources.

In this paper, electroencephalography (EEG) measurements are used to assess cortical functional connectivity in response to perceived audio quality. Specifically, in the conducted experiment the brainwave response patterns of human subjects are directly recorded using a high resolution EEG while they listen to audio whose quality varies with time. A new causal bi-directional information (CBI) measure is proposed which quantifies the information flow between EEG electrodes by appropriately grouping them into specific regions of interest (ROIs) over the cortex. It is shown that CBI can be intuitively interpreted as a causal bi-directional modification of directed information applied to a generalized cortical network setting, and inherently calculates the divergence of the observed data from a multiple access channel with feedback. The proposed measure is used to analyze and compare the information flow between ROI pairs for the case when the subject listens to high quality audio compared to when the subject listens to low quality audio. The results indicate that CBI is a more robust measure for inferring connectivity when compared to using standard directed information measures.

The second-order behavior of the discrete memoryless arbitrarily-varying channel is considered in the fixed error regime when the encoder and decoder share randomness that is independent from the adversarial choice of state. The dispersion (coefficient of the second-order term) is exactly characterized for most channels of interest when infinite shared randomness is allowed, and it is shown that precisely the same dispersion is achievable with only O (log n) bits of shared randomness. We also show that the dispersion is identical to that of the non-adversarial channel induced by the adversary simply choosing an i.i.d. state sequence according to the correct distribution. Further, we present some remarks on the connection to the compound channel, as well as on cost constraints for input and state sequences.

We study the problem of strong coordination of actions of two agents X and Y that communicate over a noisy communication channel such that the actions follow a given joint probability distribution. We propose two novel schemes for this noisy strong coordination problem, and derive inner bounds for the underlying strong coordination capacity region. The first scheme is a joint coordination-channel coding scheme that utilizes the randomness provided by the communication channel to reduce the local randomness required in generating the action sequence at agent Y. The second scheme exploits separate coordination and channel coding where local randomness is extracted from the channel after decoding. Finally, we present an example in which the joint scheme is able to outperform the separate scheme in terms of coordination rate.

We study the problem of achieving strong secrecy over wiretap channels at negligible cost, in the sense of maintaining the overall communication rate of the same channel without secrecy constraints. Specifically, we propose and analyze two source-channel coding architectures, in which secrecy is achieved by multiplexing public and confidential messages. In both cases, our main contribution is to show that secrecy can be achieved without compromising communication rate and by requiring only randomness of asymptotically vanishing rate. Our first source-channel coding architecture relies on a modified wiretap channel code, in which randomization is performed using the output of a source code. In contrast, our second architecture relies on a standard wiretap code combined with a modified source code termed uniform compression code, in which a small shared secret seed is used to enhance the uniformity of the source code output. We carry out a detailed analysis of uniform compression codes and characterize the optimal size of the shared seed.

Recently, it has been shown that a class of spatially coupled low-density generator-matrix (SC-LDGM) code ensembles displays distortion saturation for the lossy binary symmetric source coding problem with the belief propagation guided decimation (BPGD) algorithm, i.e., the BPGD distortion approaches the optimal expected distortion of the underlying ensemble asymptotically in code length. Here, we investigate the distortion performance of a practical class of protograph-based SC-LDGM code ensembles and demonstrate distortion saturation numerically. Moreover, we propose an efficient windowed encoding (WE) algorithm that takes advantage of the convolutional structure of the SC-LDGM codes. By using the WE algorithm, a distortion very close to the rate-distortion limit can be achieved for a fixed compression rate with low-to-moderate encoding latency.

This paper explores the relationship between two ideas in network information theory: edge removal and strong converses. Edge removal properties state that if an edge of small capacity is removed from a network, the capacity region does not change too much. Strong converses state that, for rates outside the capacity region, the probability of error converges to 1. Various notions of edge removal and strong converse are defined, depending on how edge capacity and residual error probability scale with blocklength, and relations between them are proved. In particular, each class of strong converse implies a specific class of edge removal. The opposite direction is proved for deterministic networks, and some discussion is given for the noisy case.

We investigate the construction of weakly-secure index codes for a sender to send messages to multiple receivers with side information in the presence of an eavesdropper. We derive a sufficient and necessary condition for the existence of index codes that are secure against an eavesdropper with access to any subset of messages of cardinality t, for any fixed t. In contrast to the benefits of using random keys in secure network coding, we prove that random keys do not promote security in three classes of index-coding instances.

It is well known that lossless compression of a discrete memoryless source with near-uniform encoder output is possible at a rate above its entropy if and only if the encoder and decoder share a common random seed. This work focuses on deriving conditions for near-uniform encoder output(s) in the Wyner-Ziv and the distributed lossy compression problems. We show that in the Wyner-Ziv problem, near-uniform encoder output and operation close to the WZ-rate limit is simultaneously possible, whereas in the distributed lossy compression problem, jointly near-uniform outputs is achievable in the interior of the distributed lossy compression rate region if the sources share non-trivial Gács-Körner common information.

We consider the problem of finding the capacity of noisy networks under the presence of Byzantine adversaries, modeled by a joint compound channel and arbitrarily varying channel (AVC) model. This extends our earlier work which considers these models only in isolation. The motivation for this setup is that typically the adversary first selects an arbitrary subset of edges from the network and then specifies adversarial transmissions to each of the selected edges. We show that in some cases equivalence between this network and another network holds in the sense that for a fixed selection of adversarial edges the noisy links can be replaced by noiseless bit-pipes with a capacity equal to the random coding capacity of the corresponding AVC. In particular, the capacity region for the noisy network can be outer bounded by the intersection of the individual capacity regions for the noiseless case, for each adversarial edge selection. Moreover, if the network is fully connected, we also show that this upper bound is equivalent to the capacity of the noisy network. We also provide necessary and sufficient condition for full connectivity, making use of a new condition for an AVC termed overwritability.

The papers in this special issue address the topic of capacity approaching codes. This issue reflects a further shift of interest in coding theory research, this time toward polar codes, a new class of capacity achieving codes introduced in 2008. Of the 17 papers appearing in this issue, 9 are devoted to various aspects of polar codes, with 6 papers devoted to LDPC codes, including 3 on spatially coupled (convolutional) LDPC codes, and 2 on other coding topics.

In this paper, we characterize the finite-length performance of separable circulant-based spatially-coupled (SCB-SC) LDPC codes for transmission over the additive white Gaussian noise channel. For a general class of finite-length graph-based codes, it is known that the existence of small absorbing sets causes a performance degradation in the error floor regime. We first present the mathematical conditions for the existence of absorbing sets in binary SCB-SC codes. This analysis enables us to find the exact number of absorbing sets as a function of the design parameters. In particular, our results show that the choice of the cutting vector affects the number of absorbing sets and, therefore, the error floor performance of the code. For a fixed column weight, we find provably optimal cutting vectors that result in the least number of absorbing sets. Furthermore, we extend our analysis to nonbinary SCB-SC codes, where we show that the choice of the cutting vector is not as critical as in the binary case. We provide an algorithm which provably removes the problematic nonbinary absorbing sets from nonbinary SCB-SC codes by informed selection of edge labels. Our simulation results show the superior error floor performance of our designed binary and nonbinary SCB-SC codes compared with binary unstructured and nonbinary quasi-cyclic SC codes available in the open literature.

We propose efficient coding schemes for two communication settings: 1) asymmetric channels and 2) channels with an informed encoder. These settings are important in non-volatile memories, as well as optical and broadcast communication. The schemes are based on non-linear polar codes, and they build on and improve recent work on these settings. In asymmetric channels, we tackle the exponential storage requirement of previously known schemes that resulted from the use of large Boolean functions. We propose an improved scheme that achieves the capacity of asymmetric channels with polynomial computational complexity and storage requirement. The proposed non-linear scheme is then generalized to the setting of channel coding with an informed encoder using a multicoding technique. We consider specific instances of the scheme for flash memories that incorporate error-correction capabilities together with rewriting. Since the considered codes are non-linear, they eliminate the requirement of previously known schemes (called polar write-once-memory codes) for shared randomness between the encoder and the decoder. Finally, we mention that the multicoding scheme is also useful for broadcast communication in Marton's region, improving upon previous schemes for this setting.

A secret sharing scheme is a method to store information securely and reliably. Particularly, in a threshold secret sharing scheme, a secret is encoded into n shares, such that any set of at least t1shares suffice to decode the secret, and any set of at most t21shares reveal no information about the secret. Assuming that each party holds a share and a user wishes to decode the secret by receiving information from a set of parties; the question we study is how to minimize the amount of communication between the user and the parties. We show that the necessary amount of communication, termed “decoding bandwidth”, decreases as the number of parties that participate in decoding increases. We prove a tight lower bound on the decoding bandwidth, and construct secret sharing schemes achieving the bound. Particularly, we design a scheme that achieves the optimal decoding bandwidth when d parties participate in decoding, universally for all t1≤ d ≤ n. The scheme is based on a generalization of Shamir's secret sharing scheme and preserves its simplicity and efficiency. In addition, we consider the setting of secure distributed storage where the proposed communication efficient secret sharing schemes not only improve decoding bandwidth but further improve disk access complexity during decoding.

In the infinite blocklength regime, spatially-coupled LDPC codes are capable of achieving capacity-approaching performance under message-passing decoding. In the finite blocklength regime, it is known that absorbing sets compete with the codewords to be the output of sub-optimal message-passing decoders: the existence of such sets in the Tanner graph of LDPC codes causes performance degradation in the low error rate region. This paper presents a mathematical approach to finding the exact number of absorbing sets in array-based spatially-coupled (AB-SC) codes. Our analysis is universal in the sense that it is in principle applicable to absorbing sets of any size. Moreover, all design parameters of AB-SC codes such as the coupling length, the circulant size, and the cutting vector are considered in the presented count. Based on our analysis, we present an approach to find provably minimal cutting vectors, with respect to the number of absorbing sets, for the construction of AB-SC codes with various circulant sizes. Simulation results show the superior error floor performance of AB-SC codes with the minimal cutting vector compared to AB-SC codes with randomly-selected cutting vectors. We also provide the average number of non-binary absorbing sets in the Tanner graph of non-binary AB-SC codes constructed by uninformed (random) assignment of edge weights to a binary AB-SC code.

We design low-complexity polar codes for empirical and strong coordination in two-node network. Our constructions hinge on the observation that polar codes may be used to approximate distribution; which we leverage to prove that nested polar codes achieve the capacity region of empirical coordination and strong coordination.

We show that solving a multiple-unicast network coding problem can be reduced to solving a single-unicast network error correction problem, where an adversary may jam at most a single edge in the network. Specifically, we present an efficient reduction that maps a multiple-unicast network coding instance to a network error correction instance while preserving feasibility. The reduction holds for both the zero probability of error model and the vanishing probability of error model. Previous reductions are restricted to the zero-error case. As an application of the reduction, we present a constructive example showing that the single-unicast network error correction capacity may not be achievable, a result of separate interest.

It is known that a sub-linear rate of source-independent random seed (common randomness) can enable the construction of lossless compression codes whose output is nearly uniform under the variational distance (Chou-Bloch-ISIT'13). This work uses finite-blocklength techniques to present an alternate proof that for near-uniform lossless compression, the seed length has to grow strictly larger than √n, where n represents the blocklength of the lossless compression code. In the lossy setting, we show the surprising result that a seed is not required to make the encoder output nearly uniform.

In this paper, we use mutual information (MI) as a measure to quantify the subjective perception of audio quality by directly measuring the brainwave responses of human subjects using a high resolution electro-encephalogram (EEG). Specifically, we propose an information theoretic model to interpret the entire “transmission chain” comprising stimulus generation, brain processing by the human subject, and EEG measurements as a nonlinear, time-varying communication channel with memory. In the conducted experiment, subjects were presented with audio whose quality varies between two quality levels. The recorded EEG measurements can be modeled as a multidimensional Gaussian mixture model (GMM). In order to make the computation of the MI feasible, we present a novel approximation technique for the differential entropy of the multidimensional GMM. We find the proposed information theoretic approach to be successful in quantifying audio quality perception, with the results being consistent across different subjects and distortion types.

We consider the noisy write-once memory (WOM) model to capture the behavior of data-storage devices such as flash memories. The noisy WOM is an asymmetric channel model with non-causal state information at the encoder. We show that a nesting of non-linear polar codes achieves the corresponding Gelfand-Pinsker bound with polynomial complexity.

We consider the problem of secure communication over a network in the presence of wiretappers. We give a new cut-set bound on secrecy capacity which takes into account the contribution of both forward and backward edges crossing the cut, and the connectivity between their endpoints in the rest of the network. We show the bound is tight on a class of networks, which demonstrates that it is not possible to find a tighter bound by considering only cut-set edges and their connectivity.

We address the problem of finding the capacity of noisy networks with either independent point-to-point compound channels (CC) or arbitrarily varying channels (AVC). These channels model the presence of a Byzantine adversary, which controls a subset of links or nodes in the network. We derive equivalence results showing that these point-to-point channels with state can be replaced by noiseless bit-pipes without changing the network capacity region. Exact equivalence results are found for the CC model, and for some instances of the AVC, including all nonsymmetrizable AVCs. These results show that a feedback path between the output and input of a CC can increase the equivalent capacity, and that if common randomness can be established between the terminals of an AVC (either by a feedback, a forward path, or via a third-party node), then again the equivalent capacity can increase. This leads to an observation that deleting an edge of arbitrarily small capacity can cause a significant change in network capacity. We also analyze an example involving an AVC for which no fixed-capacity bit-pipe is equivalent.

We study the problem of strong coordination in a three-terminal relay network, in which agents communicate to ensure that their actions follow a joint behavior specified by a prescribed joint distribution of actions. The model unifies several coordination schemes, including line and broadcast coordination. We derive several inner bounds to the strong capacity region; in particular, we prove the achievability of a subset of coordination rate-tuples, which provides insight into the relative performance of line, broadcast, and relay coordination.

We investigate channel resolvability for the l-user multiple-access channel (MAC) with two different families of encoders. The first family consists of invertible extractors, while the second one consists of injective group homomorphisms, and was introduced by Hayashi for the point-to-point channel resolvability. The main benefit of these two families is to provide explicit low-complexity channel resolvability codes in the case of symmetric MACs. Specifically, we provide two examples of families of invertible extractors suitable for MAC resolvability with uniform input distributions, one based on finite-field multiplication, which can be implemented in O(n log n) for a limited range of values of the encoding blocklength n, and a second based on modified Toeplitz matrices, which can be implemented in O(n log n) for a wider range of values of n. We also provide an example of family of injective group homomorphisms based on finite-field multiplication suitable for MAC resolvability with uniform input distributions, which can be implemented in O(n log n) for some values of n.

In this work, we first provide the definition of absorbing sets for linear channel codes over non-binary alphabets. In a graphical representation of a non-binary channel code, an absorbing set can be described by a collection of topological and edge labeling conditions. In the non-binary case, the equations relating neighboring variable and check nodes are over a non-binary field, and the edge weights are given by the non-zero elements of that non-binary field. As a consequence, it becomes more difficult for a given structure to satisfy the absorbing set constraints compared to the binary case. This observation in part explains the superior performance of non-binary codes over their binary counterparts. We show that the conditions in the non-binary absorbing set definition can be simplified in the case of non-binary elementary absorbing sets. Based on these simplified conditions, we provide design guidelines for finite-length non-binary codes free of small non-binary elementary absorbing sets. These guidelines demonstrate that even under the preserved topology, the performance of a non-binary graph-based code in the error floor region can be substantially improved by manipulating edge weights so as to avoid small absorbing sets. Our various simulation results suggest that the proposed non-binary absorbing set definition is useful for a range of code constructions and decoders. Finally, by using both insights from graph theory and combinatorial techniques, we establish the asymptotic distribution of non-binary elementary absorbing sets for regular code ensembles.

In a network application, channel coding alone is not sufficient to reliably transmit a message of finite length K from a source to one or more destinations as in, e.g., file transfer. To ensure that no data is lost, it must be combined with rateless erasure correcting schemes on a higher layer, such as a time-division multiple access (TDMA) system paired with automatic repeat request (ARQ) or random linear network coding (RLNC). We consider binary channel coding on a binary symmetric channel (BSC) and q-ary RLNC for erasure correction in a star network, where Y sources send messages to each other with the help of a central relay. In this scenario RLNC has been shown to have a throughput advantage over TDMA schemes as K→∞ and q→∞. In this paper we focus on finite block lengths and compare the expected throughputs of RLNC and TDMA. For a total message length of K bits, which can be subdivided into blocks of smaller size prior to channel coding, we obtain the channel code rate and the number of blocks that maximize the expected throughput of both RLNC and TDMA, and we find that TDMA is more throughput-efficient for small message lengths K and small q.

This work provides a generalization of absorbing sets for linear channel codes over non-binary alphabets. In a graphical representation of a non-binary channel code, an absorbing set can be described by a collection of topological and edge labeling conditions. In the non-binary case the equations relating neighboring variable and check nodes are over a non-binary field, and the edge weights are given by the non-zero elements of that non-binary field. As a consequence, it becomes more difficult for a given structure to satisfy the absorbing set constraints. This observation in part explains the superior performance of non-binary codes over their binary counterparts. We first show that, as the field order size increases, the ratio of trapping sets that satisfy the structural conditions of absorbing sets decreases. This suggests that a trapping set-only performance estimation of non-binary codes may not be as accurate in the error floor/high reliability regime. By using both insights from graph theory and combinatorial techniques, we establish the asymptotic distribution of non-binary elementary absorbing sets for regular code ensembles. Finally, we provide design guidelines for finite-length non-binary codes free of small absorbing sets.

We study the problem of strong coordination in a three-terminal line network, in which agents use common randomness and communicate over a line network to ensure that their actions follow a prescribed behavior, modeled by a target joint distribution of actions. We provide inner and outer bounds to the coordination capacity region, and show that these bounds are partially optimal. We leverage this characterization to develop insight into the interplay between communication and coordination. Specifically, we show that common randomness helps achieve optimal communication rates between agents, and that matching the network topology to the behavior structure may reduce inter-agent communication rates.

Channel coding alone is not sufficient to reliably transmit a message of finite length from a source to one or more destinations as in, e.g., file transfer. To ensure that no data is lost, it must be combined with rateless erasure correcting schemes on a higher layer, such as a time-division multiple access (TDMA) system paired with automatic repeat request (ARQ) or random linear network coding (RLNC). We consider binary channel coding on a binary symmetric channel (BSC) and q-ary RLNC for erasure correction in a star network, where Y sources send messages to each other with the help of a central relay. We focus on finite block lengths and compare the expected throughputs of RLNC and TDMA. For a total message length of K bits, which can be subdivided into blocks of smaller size prior to channel coding, we obtain the channel coding rate and the number of blocks that maximize the expected throughput of both RLNC and TDMA, and we find that TDMA is more throughput-efficient for small K and small q.

We address the problem of reliably transmitting information through a network where the nodes perform random linear network coding and where an adversary potentially injects malicious packets into the network. A good model for such a channel is a linear operator channel, where in this work we employ a combined multiplicative and additive matrix channel. We show that this adversarial channel behaves like a subspace-based symmetric discrete memoryless channel (DMC) under subspace insertions and deletions and typically has an input alphabet with non-prime cardinality. This facilitates the recent application of channel polarization results for DMCs with arbitrary input alphabets by providing a suitable one-to-one mapping from input matrices to subspaces. As a consequence, we show that polarization for this adversarial linear operator channel can be obtained via an element-wise encoder mapping for the input matrices, which replaces the finite field summation in the channel combining step for Arikan's classical polar codes.

In this paper we provide necessary and sufficient conditions for constituent codes in (multiple) concatenated and graph-based coding schemes to achieve an asymptotically error-free iterative decoding fixed-point if the maximum possible a priori information is available. At least one constituent code in an iterative decoding scheme must satisfy these conditions in order to ensure an asymptotically vanishing bit error probability at the convergence point of the decoder. Our results are proved for arbitrary binary-input symmetric memoryless channels (BISMCs) and thus can be universally applied to many transmission scenarios. Specifically, using a factor graph framework, it is shown that non-inner codes in a serial concatenation or check nodes in generalized LDPC codes achieve perfect extrinsic information if and only if the minimum Hamming distance between codewords is two or greater. For inner codes in a serial concatenation, constituent codes in a parallel concatenation, or variable nodes in doubly-generalized LDPC codes the corresponding encoder condition for acquiring perfect extrinsic information is an infinite codeword weight for a weight-one input sequence. For this case we provide a general proof which holds for all linear encoders and BISMCs. We also show that these results can improve the performance of concatenated coding schemes.

The secrecy capacity of a network, for a given collection of permissible wiretap sets, is the maximum rate of communication such that observing links in any permissible wiretap set reveal no information about the message. This paper considers secure network coding with nonuniform or restricted wiretap sets, for example, networks with unequal link capacities where a wiretapper can wiretap any subset ofklinks, or networks where only a subset of links can be wiretapped. Existing results show that for the case of uniform wiretap sets (networks with equal capacity links/packets where anykcan be wiretapped), the secrecy capacity is given by a cut-set bound if random keys are injected at the source (and decoded at the sink), whether or not the communicating users have information about the choice of wiretap set. In contrast, we show that for the nonuniform case, this secrecy rate is achievable for the case of known but not unknown wiretap set. We give achievable linear optimization-based strategies where random keys are canceled at intermediate nonsink nodes or injected at intermediate nonsource nodes. Finally, we show that determining the secrecy capacity is an NP-hard problem.

We consider transmission over a wiretap channel where both the main channel and the wiretapper's channel are binary erasure channels (BEC). A code construction method is proposed using two edge-type low-density parity-check (LDPC) codes based on the coset encoding scheme. Using a single edge-type LDPC ensemble with a given threshold over the BEC, we give a construction for a two edge-type LDPC ensemble with the same threshold. If the given single edge-type LDPC ensemble has degree two variable nodes, our construction gives rise to degree one variable nodes in the code used over the main channel. This results in zero threshold over the main channel. In order to circumvent this problem, the degree distribution of the two edge-type LDPC ensemble is numerically optimized. We find that the resulting ensembles are able to perform close to the boundary of the rate-equivocation region of the wiretap channel. Further, a method to compute the ensemble average equivocation of two edge-type LDPC ensembles is provided by generalizing a recently published approach to measure the equivocation of single edge-type ensembles for transmission over the BEC in the point-to-point setting. From this analysis, we find that relatively simple constructions give very good secrecy performance.

This work provides an algebraic framework for source coding with decoder side information and its dual problem, channel coding with encoder side information, showing that nested concatenated codes can achieve the corresponding rate-distortion and capacity-noise bounds. We show that code concatenation preserves the nested properties of codes and that only one of the concatenated codes needs to be nested, which opens up a wide range of possible new code combinations for these side information based problems. In particular, the practically important binary version of these problems can be addressed by concatenating binary inner and non-binary outer linear codes. By observing that list decoding with folded Reed-Solomon codes is asymptotically optimal for encoding IID q-ary sources and that in concatenation with inner binary codes it can asymptotically achieve the rate-distortion bound for a Bernoulli symmetric source, we illustrate our findings with a new algebraic construction which comprises concatenated nested cyclic codes and binary linear block codes.

In this paper, we investigate how constraints on the randomization in the encoding process affect the secrecy rates achievable over wiretap channels. In particular, we characterize the secrecy capacity with a rate-limited local source of randomness and a less capable eavesdropper's channel, which shows that limited rate incurs a secrecy rate penalty but does not preclude secrecy. We also show that secure communication is possible when randomizing with a non-uniform source of randomness, which suggests the possibility of designing robust coding schemes.

We consider communication protocols for N-way all-cast relay networks, which comprise N source terminals such that each source terminal demands messages from all other source terminals with the help of a relay. The derived protocols are characterized by the fact that physical layer network coding is employed at the relay, where each source has side information about the signals it has sent. Amplify-and-forward (AF) and decode-and-forward (DF) protocols are applied to the N-way relay network setting, where the achievable rate regions for those protocols are derived and compared with outer capacity bounds. We propose several practical space-time coding schemes for AF and DF, and introduce two new protocols denoted as denoise-and-forward (DNF) and estimate-and-forward (EF). Further, for AF and DF the fundamental diversity-multiplexing trade-off is characterized.

We investigate the design of network codes for multiple-user multiple-relay (MUMR) wireless networks with slow fading (quasi-static) channels. In these networks, M users have independent information to be transmitted to a common base station (BS) with the help of N relays, where M ≥ 2 and N ≥ 1 are arbitrary integers. We investigate such networks in terms of diversity order to measure asymptotic performance. For networks with orthogonal channels, we show that network codes based on maximum distance separable (MDS) codes can achieve the maximum diversity order of N+1. We further show that the MDS coding construction of network codes is also necessary to obtain full diversity for linear finite field network coding (FFNC). Then, we compare the performance of the FFNC approach with superposition coding (SC) at the relays. The results show that the FFNC based on MDS codes has better performance than SC in both the high rate and the high SNR regime. Further, we discuss networks without direct source-to-BS channels for N ≥ M. We show that the proposed FFNC can obtain the diversity order N-M+1, which is equivalent to achieving the Singleton bound for network error-correction codes. Finally, we study the network with nonorthogonal channels and show our codes can still achieve a diversity order of N+1, which cannot be achieved by a scheme based on SC.

It has been widely observed that there exists a fundamental tradeoff between the minimum (Hamming) distance properties and the iterative decoding convergence behavior of turbo-like codes. While capacity-achieving code ensembles typically are asymptotically bad in the sense that their minimum distance does not grow linearly with block length, and they therefore exhibit an error floor at moderate-to-high signal-to-noise ratios, asymptotically good codes usually converge further away from channel capacity. In this paper, we introduce the concept of tuned turbo codes, a family of asymptotically good hybrid concatenated code ensembles, where asymptotic minimum distance growth rates, convergence thresholds, and code rates can be tradedoff using two tuning parameters:$\lambda $and$\mu $. By decreasing$\lambda $, the asymptotic minimum distance growth rate is reduced in exchange for improved iterative decoding convergence behavior, while increasing$\lambda $raises the asymptotic minimum distance growth rate at the expense of worse convergence behavior, and thus, the code performance can be tuned to fit the desired application. By decreasing$\mu $, a similar tuning behavior can be achieved for higher rate code ensembles.

We consider a line of terminals which is connected by packet erasure channels and where random linear network coding is carried out at each node prior to transmission. In particular, we address an online approach in which each terminal has local information to be conveyed to the base station at the end of the line and provide a queueing theoretic analysis of this scenario. First, a genie-aided scenario is considered and the average delay and average transmission energy depending on the link erasure probabilities and the Poisson arrival rates at each node are analyzed. We then assume that all nodes cannot send and receive at the same time. The transmitting nodes in the network send coded data packets before stopping to wait for the receiving nodes to acknowledge the number of degrees of freedom, if any, that are required to decode correctly the information. We analyze this problem for an infinite queue size at the terminals and show that there is an optimal number of coded data packets at each node, in terms of average completion time or transmission energy, to be sent before stopping to listen.

Channel coding alone is not sufficient to reliably transmit a message of finite length from a source to one or more destinations. To ensure that no data is lost, channel coding on the physical layer needs to be combined with rateless erasure correcting schemes such as automatic repeat request (ARQ) or random linear network coding (RLNC) on a higher layer. In this paper we consider channel coding on a binary symmetric channel and random linear network coding for erasure correction. Given a message of length K and network coding over a finite Galois field of size q, we obtain the optimal number of blocks for network coding that minimizes the expected number of transmissions. We consider both a single link and broadcast to n destinations. As the field size of network coding gets large and the expected coding overhead in blocks becomes small, we show that, given our assumptions, the benefit of using a larger channel coded block outweighs the advantage of employing network coding over many blocks and the optimal number of number of blocks tends to one, making RLNC equivalent to simple ARQ.

This work considers the multiple-access multicast error-correction scenario over a packetized network withzmalicious edge adversaries. The network has min-cutmand packets of lengthl, and each sink demands all information from the set of sourcesS. The capacity region is characterized for both a “side-channel” model (where sources and sinks share some random bits that are secret from the adversary) and an “omniscient” adversarial model (where no limitations on the adversary's knowledge are assumed). In the “side-channel” adversarial model, the use of a secret channel allows higher rates to be achieved compared to the “omniscient” adversarial model, and a polynomial-complexity capacity-achieving code is provided. For the “omniscient” adversarial model, two capacity-achieving constructions are given: the first is based on random subspace code design and has complexity exponential inlm, while the second uses a novel multiple-field-extension technique and has O(lm|S|) complexity, which is polynomial in the network size. Our code constructions are “end-to-end” in that all nodes except the sources and sinks are oblivious to the adversaries and may simply implement predesigned linear network codes (random or otherwise). Also, the sources act independently without knowledge of the data from other sources.

We introduce a family of binary prefix condition codes in which each codeword is required to have a Hamming weight which is a multiple of w for some integer w ¿ 2. Such codes have intrinsic error resilience and are a special case of codes with codewords constrained to belong to a language accepted by a deterministic finite automaton. For a given source over n symbols and parameter w we offer an algorithm to construct a minimum-redundancy code among this class of prefix condition codes which has a running time of O(nw+2).

Nested codes have been employed in a large number of communication applications as a specific case of superposition codes, for example to implement binning schemes in the presence of noise, in joint network-channel coding, or in physical-layer secrecy. Whereas nested lattice codes have been proposed recently for continuous-input channels, in this paper we focus on the construction of nested linear codes for joint channel-network coding problems based on algebraic photograph LDPC codes. In particular, over the past few years several constructions of codes have been proposed that are based on random lifts of suitably chosen base graphs. More recently, an algebraic analog of this approach was introduced using the theory of voltage graphs. In this paper we illustrate how these methods can be used in the construction of nested codes from algebraic lifts of graphs.

We address secure network coding over networks with unequal link capacities in the presence of a wiretapper who has only access to a restricted number of k links in the network. Previous results show that for the case of equal link capacities and unrestricted wiretapping sets, the secrecy capacity is given by the cut-set bound, whether or not the location of the wiretapped links is known. The cut-set bound can be achieved by injecting k random keys at the source which are decoded at the sink along with the message. In contrast, for the case where the wiretapping set is restricted, or where link capacities are not equal, we show that the cut-set bound is not achievable in general. Finally, it is shown that determining the secrecy capacity is a NP-hard problem.

We analyze the achievable rate in interference free wireless networks with physical layer fading channels and orthogonal multiple access. As a starting point, the point-to-point channel is considered. We find the optimal physical and network layer rate trade-off which maximizes the achievable overall rate for both a fixed rate transmission scheme and an improved scheme based on multiple virtual users and superposition coding. These initial results are extended to the network setting, where, based on a cut-set formulation, the achievable rate at each node and its upper bound are derived. We propose a distributed optimization algorithm which allows to jointly determine the maximum achievable rate, the optimal physical layer rates on each network link, and an opportunistic back-pressure-type routing strategy on the network layer. This inherently justifies the layered architecture in existing wireless networks. Finally, we show that the proposed layered optimization approach can achieve almost all of the ergodic network capacity in high SNR.

The serial concatenation of a repetition code with two or more accumulators has the advantage of a simple encoder structure. Furthermore, the resulting ensemble is asymptotically good and exhibits minimum distance growing linearly with block length. However, in practice these codes cannot be decoded by a maximum likelihood decoder, and iterative decoding schemes must be employed. For low-density parity-check codes, the notion of trapping sets has been introduced to estimate the performance of these codes under iterative message passing decoding. In this paper, we present a closed form finite length ensemble trapping set enumerator for repeat multiple accumulate codes by creating a trellis representation of trapping sets. We also obtain the asymptotic expressions when the block length tends to infinity and evaluate them numerically.

In this paper we derive capacity regions for network error correction with both known and unknown topologies (coherent and non-coherent network coding) under a multiple-source multicast transmission scenario. For the multiple-source non-multicast scenario, given any achievable network code for the error-free case, we construct a code with a reduced rate region for the case with errors.

We propose relaying strategies for uncoded two-way relay channels, where two terminals transmit simultaneously to each other with the help of a relay. In particular, we consider a memoryless system, where the signal transmitted by the relay is obtained by applying an instantaneous relay function to the previously received signal. For binary antipodal signaling, a class of so called absolute (abs)-based schemes is proposed in which the processing at the relay is solely based on the absolute value of the received signal. We analyze and optimize the symbol-error performance of existing and new abs-based and non-abs-based strategies under an average power constraint, including abs-based and non-abs-based versions of amplify and forward (AF), detect and forward (DF), and estimate and forward (EF). Additionally, we optimize the relay function via functional analysis such that the average probability of error is minimized at the high signal-to-noise ratio (SNR) regime. The optimized relay function is shown to be a Lambert W function parameterized on the noise power and the transmission energy. The optimized function behaves like abs-AF at low SNR and like abs-DF at high SNR, respectively; EF behaves similarly to the optimized function over the whole SNR range. We find the conditions under which each class of strategies is preferred. Finally, we show that all these results can also be generalized to higher order constellations.

We present a novel variable-length code (VLC) construction which exhibits an inherent error correcting capability due to the exclusive presence of codewords with even Hamming weight. Besides error robustness, the proposed code construction features a similar codeword length distribution as Golomb-Rice codes, and therefore, in particular for sources with exponentially distributed symbols, has good source compression properties at the same time. We show that in a source channel coding framework with outer source encoding, inner channel encoding with a recursive convolutional code, and iterative decoding the proposed VLC construction can lead to significant performance improvements compared to fixed-length source encoding with optimized mappings. In particular, simulation results for the AWGN channel verify that for Gauss-Markov sources a performance close to the theoretical limit can be achieved.

This paper analyzes the error performance of a coded cooperative diversity system employing the Euclidean superposition of two BPSK-modulated signals. For an example using a convolutional code on block fading channels, the results show excellent agreement with computer simulations. The analysis makes it possible to optimize the power allocation between the local and relay signals numerically, circumventing the need for time consuming Monte Carlo simulations. Similarly, the analysis demonstrates how the power allocation can be "tuned" to compensate for unbalanced uplink channels and/or to provide unequal error protection to the data from the two cooperating nodes.

The design of double serially concatenated convolutional codes with S-type permutors, i.e., permutors that provide a nontrivial separation, is considered. Based on a newly introduced parameter, namely, the so-called symbol span, a joint design of the outer and inner permutor is presented and its impact on the minimum distance of the overall code is analyzed. It is shown that a lower bound on the minimum distance that is given by the product of the free distances of all three component codes can be guaranteed. Design tables and simulation results are presented that include comparisons with single serially concatenated convolutional codes. In addition, a comparison with double/generalized repeat accumulate codes is briefly sketched.

We address communication protocols for N-way relay networks with M antennas at the relay and a single antenna at the N source terminals. In particular, amplify-and-forward (AF), decode-and-forward (DF), and compress-and-forward (CF) strategies are extended to these networks, and in addition, two new relaying protocols, denoise-and-forward and estimate-and-forward, are proposed. In the first part of the paper, the performance of these schemes is analyzed in terms of the achievable rate region. Also, the optimal diversity-multiplexing tradeoff is derived for both AF and DF. The second part of the paper is devoted to practical space-time transmission strategies. Linear dispersion codes are used, which are optimized by maximizing the sum rate. For AF a diversity order of close to M can be achieved by using a specific space-time code construction.

We consider relaying strategies for two-way relay channels, where two terminals transmits simultaneously to each other with the help of relays. A memoryless system is considered, where the signal transmitted by a relay depends only on its last received signal. For binary antipodal signaling, we analyze and optimize the performance of existing amplify and forward (AF) and absolute (abs) decode and forward (ADF) for two- way AWGN relay channels. A new abs-based AF (AAF) scheme is proposed, which has better performance than AF. In low SNR, AAF performs even better than ADF. Furthermore, a novel estimate and forward (EF) strategy is proposed which performs better than ADF. More importantly, we optimize the relay strategy within the class of abs-based strategies via functional analysis, which minimizes the average probability of error over all possible relay functions. The optimized function is shown to be a Lambert's W function parameterized on the noise power and the transmission energy. The optimized function behaves like AAF in low SNR and like ADF in high SNR, resp., where EF behaves like the optimized function over the whole SNR range.

It has recently been shown that the minimum distance of the ensemble of repeat multiple accumulate codes grows linearly with block length. In this paper, we present a method to obtain the distance growth rate coefficient of multiple-serially concatenated code ensembles and determine the growth rate coefficient of the rate 1/2 double-serially concatenated code consisting of an outer memory one convolutional code followed by two accumulators. We compare both the growth rate of the minimum distance, as well as the convergence behavior, of this code with rate 1/2 repeat multiple accumulate codes, and we show that repeat multiple accumulate codes have better minimum distance growth but worse performance in terms of convergence.

Bandwidth efficient parallel-concatenated Turbo Trellis Coded Modulation (TTCM) schemes were designed for communicating over uncorrelated Rayleigh fading channels. A symbol-based union bound was derived for analysing the error floor of the proposed TTCM schemes. A pair of In-phase (I) and Quadrature-phase (Q) interleavers were employed for interleaving the I and Q components of the TTCM coded symbols, in order to attain an increased diversity gain. The decoding convergence of the IQ-TTCM schemes was analysed using symbol-based EXtrinsic Information Transfer (EXIT) charts. The best TTCM component codes were selected with the aid of both the symbolbased union bound and non-binary EXIT charts, for designing capacity-approaching IQ-TTCM schemes in the context of 8 PSK, 16 QAM, 32 QAM and 64 QAM modulation schemes.

This paper considers a simple network consisting of a source, a destination, and a relay. In this model, the source- relay and relay-destination links are lossless, while the source- destination link is subject to erasures. Four coding schemes for reliably conveying k symbols from the source to the destination are described. Three of these techniques are adapted directly from well-known point-to-point coding schemes - viz., the use of maximum-distance separable (MDS) codes and Luby Transform (LT) codes. The fourth approach is a new technique using uncoded transmission from the source in conjunction with a relay that transmits a sequence with this property: When the destination subtracts the effects of the unerased symbols from the sequence, what remains is an "LT-like" code for the erased symbols - and this property holds regardless of which symbols were erased on the source-destination link. The four approaches are compared in terms of their complexity and performance.

This paper presents a technique for achieving cooperative spatial diversity using serially concatenated low density generator matrix (LDGM) codes. Specifically, we consider a scenario in which a pair of transceivers employ algebraic superposition of error control codes to effect spatial diversity at their common destination. The construction of LDGM codes from a sparse generator matrix makes them a natural fit for such a cooperative diversity scheme. The simple decoder structure for graph based codes reduces the complexity at the destination compared with previously-proposed schemes using algebraic superposition of convolutional codes and turbo-like decoding. The result is a system with low encoding and decoding complexity and improved error performance.

For wireless fading networks where M source nodes wish to multicast their information to N destination nodes via a common relay, it has been shown that network coding at the relay can reduce the number of required transmissions, if the destination nodes overhear M - 1 source nodes transmitting to the relay. Nested coding is a recently proposed alternative where, unlike network coding, physical and network layers are jointly designed. We find that for a given network throughput ideal nested coding in many situations can lead to a significant increase in transmission reliability compared to network coding. We also consider the case of feedback and retransmissions when received packets fail to be decoded. For the case of two source nodes it is shown that by using nested codes the expected number of relay transmissions is reduced. However, for a larger number of source nodes the gain depends on the employed nested coding strategy at the relay.

This paper describes techniques to decompose LT codes (a class of rateless erasure-correcting codes) into distributed LT (DLT) codes. DLT codes can be used to independently encode data from multiple sources in a network in such a way that, when the DLT-encoded packets are combined at a common relay, the resulting bit stream (called a modified LT (MLT) code) has a degree distribution approximating that of an LT code, with simulations indicating comparable performance. In essence, DLT codes are designed so that the final stage of encoding for erasure correction can be carried out by a low-complexity relay that selectively xors the bit streams generated at each source and transmits the result to the sink. This paper presents results for two-source and four-source networks. It is shown that, when the relay-to-sink link is the bottleneck, the DLT/MLT approach can yield substantial performance benefits compared with a competing strategy wherein each of the sources uses its own independent LT encoder and the resulting bit streams are time-multiplexed through the relay.

This paper proposes a network coding approach to cooperative diversity featuring the algebraic superposition of channel codes over a finite field. The scenario under consideration is one in which two ldquopartnersrdquo - node A and node B - cooperate in transmitting information to a single destination; each partner transmits both locally generated information and relayed information that originated at the other partner. A key observation is that node B already knows node A's relayed information (because it originated at node B) and can exploit that knowledge when decoding node A's local information. This leads to an encoding scheme in which each partner transmits the algebraic superposition of its local and relayed information, and the superimposed codeword is interpreted differently at the two receivers i.e., at the other partner and at the destination node, based on their different a priori knowledge. Decoding at the destination is then carried out by iterating between the codewords from the two partners. It is shown via simulation that the proposed scheme provides substantial coding gain over other cooperative diversity techniques, including those based on time multiplexing and signal (Euclidean space) superposition.

This paper describes an efficient implementation of binning for decode-and-forward (DF) in relay channels using low-density parity-check (LDPC) codes. Bilayer LDPC codes are devised to approach the theoretically promised rate of the DF relaying strategy by incorporating relay-generated parity bits in specially designed bilayer graphical code structures. While conventional LDPC codes are sensitively tuned to operate efficiently at a certain channel parameter, the proposed bilayer LDPC codes are capable of working at two different channel parameters and two different rates: that at the relay and at the destination. To analyze the performance of bilayer LDPC codes, bilayer density evolution is devised as an extension of the standard density evolution algorithm. Based on bilayer density evolution, a design methodology is developed for the bilayer codes in which the degree distribution is iteratively improved using linear programming. Further, in order to approach to the theoretical DF rate for a wide range of channel parameters, this paper proposes two different forms of bilayer codes: the bilayer-expurgated and bilayer-lengthened codes. It is demonstrated that the rate of a properly designed bilayer LDPC code can closely approach the theoretical DF limit. Finally, it is shown that a generalized version of the proposed bilayer code construction is applicable to relay networks with multiple relays.

A joint video and channel coded system employing an iteratively decoded serial concatenation of a vector quantization (VQ) based video codec and a trellis-coded modulation (TCM) scheme is proposed. The video codec imposes VQ-induced code constraints, which may be completely described by a trellis structure, which is employed as the basis for optimal minimum mean-squared-error VQ-encoding and -decoding. In the latter case, the Bahl-Cocke-Jelinek-Raviv (BCJR) algorithm is employed to facilitate the iterative exchange of soft information between the VQ and TCM decoder. An error-free video reconstruction quality is supported using 16-Ievel quadrature amplitude modulation (16QAM) based TCM for transmission over Rayleigh-fading channels at a signal-to-noise ratio (SNR) per bit of 5.25 dB. This value is within 1.29 dB of the Rayleigh channel's capacity at our system's effective bandwidth-efficiency of 2 bits/s/Hz. Owing to its ability to exploit the VQ-induced code constraints during iterative decoding, the joint video and channel coding approach is found to consistently outperform the Shannonian source and channel separation philosophy. This is achieved at the cost of a 1.6 times higher computational complexity. Finally, the convergence of the iterative decoder is investigated with the aid of a novel so-called extrinsic information transfer (EXIT) chart

In this paper we address the extrinsic information transfer functions of inner decoders for a serially concatenated coding scheme. For the case of an AWGN channel, we give a universal proof for the fact that only inner encoders yielding an infinite output weight for a weight-one input sequence, such as recursive convolutional encoders, lead to perfect extrinsic information at the output of the corresponding SISO decoder. As an example we consider bit-interleaved coded modulation with iterative demapping (BICM-ID) and insert an additional recursive precoder prior to the mapping operation. Simulation results show that the proposed system does not suffer from an error floor and thus significantly outperforms BICM-ID systems that solely use mappings as inner encodings, even when they are optimized

This paper proposes a novel distributed encoding procedure to realize codes that resemble LT codes (rateless codes for erasure correction) in both structure and performance. For the case of two sources communicating with a single sink via a common relay, this technique separately encodes k/2 symbols of information onto slightly more than k code symbols at each source. These two codewords are then selectively XOR-ed at the relay, such that the result can be decoded by the sink to recover all k information symbols. It is shown that, for the case of four sources communicating to a single sink, the use of a similar distributed LT code leads to a 50% reduction in overhead at the sink, compared to the use of four individual LT codes

This paper proposes a new approach to cooperative diversity based on the algebraic superposition of channel codes over a finite field. The scenario under consideration is one in which two "partners" - Node A and Node B cooperate in transmitting information to a single destination; each partner transmits both locally-generated information and relayed information that originated at the other partner. A key observation is that Node B already knows Node A's relayed information (previously sent from Node B) and can exploit that knowledge when decoding Node A's local information. This leads to an encoding scheme in which each partner transmits the algebraic superposition of its local and relayed information, and the superimposed codeword is interpreted differently at the two receivers - i.e., at the other partner and at the destination node - based on their different a priori knowledge. It is shown via simulation that the proposed scheme provides substantial coding gain over other cooperative diversity techniques, including those based on time sharing and signal (Euclidean space) superposition

In this paper we design bandwidth efficient parallel-concatenated turbo trellis coded modulation (TTCM) schemes for communicating over AWGN and uncorrelated Rayleigh fading channels. The convergence properties of the symbol-based TTCM schemes employing various constituent codes, were analysed using symbol-based extrinsic information transfer (EXIT) charts. The traditional method used of generating EXIT charts is based on computationally complex multidimensional histogram measurements, which is only feasible for analysing TTCM schemes employing low-order modulation schemes, such as 4PSK and 8PSK. Hence, a novel low-complexity technique was employed in this paper for computing the symbol-based EXIT charts. Capacity-approaching TTCM schemes were designed based on the best constituent codes found when employing 8PSK, 16QAM, 32QAM and 64QAM modulation schemes.

The calculation of nonbinary extrinsic information transfer charts for the iterative decoding of concatenated index-based codes is addressed. We show that the extrinsic information at the output of a constituent a posteriori probability decoder can be calculated with very low complexity, where expensive histogram measurements are not required any more. An example for turbo trellis-coded modulation demonstrates the capabilities of the proposed approach

In this paper, we present a joint source-channel coding method which employs quantized overcomplete frame expansions that are binary transmitted through noisy channels. The frame expansions can be interpreted as real-valued block codes that are directly applied to waveform signals prior to quantization. At the decoder, first the index-based redundancy is used by a soft-input soft-output source decoder to determine the a posteriori probabilities for all possible symbols. Given these symbol probabilities, we then determine least-squares estimates for the reconstructed symbols. The performance of the proposed approach is evaluated for code constructions based on the DFT and is compared to other decoding approaches as well as to classical BCH block codes. The results show that the new technique is superior for a wide range of channel conditions, especially when strict delay constraints for the transmission system are given

In this paper, we present a novel packetized bit-level decoding algorithm for variable-length encoded Markov sources, which calculates reliability information for the decoded bits in the form of a posteriori probabilities (APPs). An interesting feature of the proposed approach is that symbol-based source statistics in the form of the transition probabilities of the Markov source are exploited as a priori information on a bit-level trellis. This method is especially well-suited for long input blocks, since in contrast to other symbol-based APP decoding approaches, the number of trellis states does not depend on the packet length. When additionally the variable-length encoded source data is protected by channel codes, an iterative source-channel decoding scheme can be obtained in the same way as for serially concatenated codes. Furthermore, based on an analysis of the iterative decoder via extrinsic information transfer charts, it can be shown that by using reversible variable-length codes with a free distance of two, in combination with rate-1 channel codes and residual source redundancy, a reliable transmission is possible even for highly corrupted channels. This justifies a new source-channel encoding technique where explicit redundancy for error protection is only added in the source encoder.

We present a novel symbol-based soft-input a posteriori probability (APP) decoder for packetized variable-length encoded source indexes transmitted over wireless channels where the residual redundancy after source encoding is exploited for error protection. In combination with a mean-square or maximum APP estimation of the reconstructed source data, the whole decoding process is close to optimal. Furthermore, solutions for the proposed APP decoder with reduced complexity are discussed and compared to the near-optimal solution. When, in addition, channel codes are employed for protecting the variable-length encoded data, an iterative source-channel decoder can be obtained in the same way as for serially concatenated codes, where the proposed APP source decoder then represents one of the two constituent decoders. The simulation results show that this iterative decoding technique leads to substantial error protection for variable-length encoded correlated source signals, especially, when they are transmitted over highly corrupted channels.

In this paper, we propose a novel iterative source-channel decoding approach for robust transmission of compressed still images over noisy communication channels. Besides the explicit redundancy introduced by channel encoding, also implicit residual source redundancy is exploited for error protection. The source redundancy is modeled by a Markov random field (MRF) source model, which considers the residual spatial correlation after source encoding. The resulting MRF-based soft-input/soft-output source decoder is used as outer constituent decoder in the proposed iterative source-channel decoding scheme, where due to the link between MRFs and the Gibbs distribution, the source decoder can be implemented with very low complexity. We show that this iterative decoding scheme can be successfully employed for recovering the image data, especially when the channel is highly corrupted.

A novel approach for robust source transmission is presented where variable-length code (VLC) source and convolutional channel encoding are concatenated in parallel. Simulation results show that the proposed scheme leads to a strong increase in the signal-to-noise ratio at the decoder output compared to a serial concatenation of VLCs and channel codes

We propose a soft-decoding method for quantized overcomplete frame expansions that are binary transmitted through noisy channels. The frame expansions can be viewed as real-valued block codes that are directly applied to waveform signals prior to quantization. The explicit redundancy introduced in the continuous amplitude domain is exploited by the decoder in two stages. First, the index-based redundancy is used by a soft-input soft-output source decoding approach that outputs decoded symbols together with their reliability information. In a second stage, the soft information on the symbols and the structure of the introduced redundancy are used to correct errors. The performance of the proposed approach is evaluated for different code constructions based on the discrete Fourier transform (DFT), the discrete cosine transform (DCT), and the discrete Hadamard transform (DHT), and is compared to standard approaches without soft decoding.

We present an improved index-based a-posteriori probability (APP) decoding approach for variable-length encoded packetized data, where implicit residual source correlation is exploited for error protection. The proposed algorithm is based on a novel generalized two-dimensional state representation which leads to a three-dimensional trellis with unique state transitions. APP decoding on this trellis is realized by employing a two-dimensional version of the classical BCJR algorithm. This new method has the advantage that, due to the unique state representation, all available a-priori information can be fully exploited, which especially holds for the transition probabilities of the Markov model associated with the variable-length encoded source indices. Simulation results for an additional error protection by channel codes and iterative joint source-channel decoding show that the proposed approach leads to an increased error-correction performance compared to previously published results where a one-dimensional state representation is used.

We utilize both the implicit residual source correlation and the explicit redundancy from a forward error correction (FEC) scheme for the error protection of packetized variable-length encoded source indices. The implicit source correlation is exploited in a novel symbol-based soft-input a-posteriori probability (APP) decoder, which leads to an optimal decoding process in combination with a mean-squares or maximum a-posteriori probability estimation of the reconstructed source signal. When, additionally, the variable-length encoded source data is protected by channel codes, an iterative source-channel decoder can be obtained in the same way as for serially concatenated codes, where the outer constituent decoder is replaced by the proposed APP source decoder. Simulation results show that, by additionally considering the correlations between the variable-length encoded source indices, the error-correction performance can be highly increased.

In this paper we address design and properties of an oversampled non-uniform DFT filter bank derived by an allpass frequency transform from its uniform version. The novel synthesis bank utilizes only stable FIR filters, which can be designed via closed-form expressions. The overall analysis-synthesis system leads to a near-perfect-reconstruction solution, where the phase compensation error can be made arbitrarily small at the expense of additional system delay. Furthermore, we also address the case of different sub-sampling factors in the subbands. The filter bank design is carried out by utilizing a lifting factorization for the prototypes, which has the advantage that the overall system delay can be controlled in an efficient way.

In this paper we discuss the application of a joint source-channel decoding approach to image transmission over wireless channels. In addition to channel codes, also the implicit residual redundancy after source encoding in both horizontal and vertical direction is utilized for error protection. At the decoder we use an iterative (“turbo”) source-channel decoder which can be obtained in the same manner as for serially concatenated channel codes. As a new result we show that this iterative decoding scheme in combination with a novel simplified joint source and channel coding rate allocation at the encoder can be successfully employed for protecting the image data, especially when the channel is highly corrupted. Furthermore, when the source correlations are approximated with a large training set at the decoder, only a small loss in performance is observed.

We address the transmission of compressed images over highly corrupted AWGN-channels using an optimal estimation approach at the decoder. In contrast to other methods, we use only a negligible amount of explicit redundancy based on channel codes. Mainly, the implicit residual source redundancy inherent in the quantized subband images and the bit-reliability information at the channel output are utilized for error protection. As a novelty, we extend the optimal estimation technique from the one- to the two-dimensional case, where both horizontal and vertical correlations are exploited in the subband images. Based on this approach, the performances for several estimation methods are compared. Approaches for approximating the source correlations at the decoder are also discussed.

In this paper we derive perfect reconstruction (PR) conditions for oversampled cosine-modulated filter banks. The results can be regarded as a generalization of the known work for critical subsampling. We show that in the oversampled case we gain some additional degree of freedom, which can be exploited in the filter design process. This leads to PR prototypes with stopband attenuations being much higher than in the critically subsampled PR case. The filters designed as PR filters for the oversampled case can also serve as prototypes for critically subsampled cosine-modulated pseudo QMF banks.

In this paper, methods for processing arbitrary-length input signals with MDFT filter banks are presented. The MDFT filter bank can be regarded as the most general type of a special class of filter banks. These filter banks have linear phase analysis filters, but different centers of symmetry due to subsampling with and without a phase shift. Already known extension methods cannot be applied to these filter banks in their original form. We first discuss the symmetric extension for two special input signal lengths. These cases are then incorporated in the general solution for arbitrary-length input signals.