This paper studies the rate-distortion-perception (RDP) tradeoff for a memoryless source model in the asymptotic limit of large block-lengths. The perception measure is based on a divergence between the distributions of the source and reconstruction sequences conditioned on the encoder output, first proposed by Mentzer et al. We consider the case when there is no shared randomness between the encoder and the decoder. For the case of discrete memoryless sources we derive a single-letter characterization of the RDP function, in contrast to the marginal-distribution metric case (introduced by Blau and Michaeli), whose RDP characterization remains open when there is no shared randomness. The achievability scheme is based on lossy source coding with a posterior reference map. For the case of continuous valued sources under squared error distortion measure and squared quadratic Wasserstein perception measure we also derive a single-letter characterization and show that a noise-adding mechanism at the decoder suffices to achieve the optimal representation. Interestingly, the RDP function characterized for the case of zero perception loss coincides with that of the marginal metric and further zero perception loss can be achieved with a 3-dB penalty in minimum distortion. Finally we specialize to the case of Gaussian sources, and derive the RDP function for Gaussian vector case and propose a waterfilling like solution. We also partially characterize the RDP function for a mixture of Gaussian vector sources.

The paper considers a communication constrained distributed hypothesis testing problem in which the transmitter sends a message about its local observation to the receiver, and the receiver tries to decide whether or not its own observation is independent of the observation at the transmitter. We analyze the problem in the one-shot setting and derive an achievability region under both the fixed-length and the variable-length communication constraints. Novel information-theoretic tools, including the generalized Poisson matching lemma and the strong functional representation lemma, are applied. It is shown that the proposed one-shot schemes, when applied to the asymptotic case, recover the optimal fixed-length and variable-length type-II error exponents for testing against independence.

In federated learning (FL), the communication constraint between the remote clients and the Parameter Server (PS) is a crucial bottleneck. For this reason, model updates must be compressed so as to minimize the loss in accuracy resulting from the communication constraint. This paper proposes “M-magnitude weighted L2 distortion + 2 degrees of freedom” (M22) algorithm, a rate-distortion inspired approach to gradient compression for federated training of deep neural networks (DNNs). In particular, we propose a family of distortion measures between the original gradient and the reconstruction we referred to as “$M$-magnitude weighted$L_{2}$” distortion, and we assume that gradient updates follow an i.i.d. distribution – generalized normal or Weibull, which have two degrees of freedom. In both the distortion measure and the gradient distribution, there is one free parameter for each that can be fitted as a function of the iteration number. Given a choice of gradient distribution and distortion measure, we design the quantizer to minimize the expected distortion in gradient reconstruction. To measure the gradient compression performance under a communication constraint, we define the per-bit accuracy as the optimal improvement in accuracy that one bit of communication brings to the centralized model over the training period. Using this performance measure, we systematically benchmark the choice of gradient distribution and distortion measure. We provide substantial insights on the role of these choices and argue that significant performance improvements can be attained using such a rate-distortion inspired compressor.

This paper studies the rate-distortion-perception (RDP) tradeoff for a memoryless source model in the asymptotic limit of large block-lengths. The perception measure is based on a divergence between the distributions of the source and reconstruction sequences conditioned on the encoder output, first proposed by Mentzer et al. We consider the case when there is no shared randomness between the encoder and the decoder and derive a single-letter characterization of the RDP function, for the case of discrete memoryless sources. This is in contrast to the marginal-distribution metric case (introduced by Blau and Michaeli), whose RDP characterization remains open when there is no shared randomness. The achievability scheme is based on lossy source coding with a posterior reference map. For the case of continuous valued sources under the squared error distortion measure and the squared quadratic Wasserstein perception measure, we also derive a single-letter characterization and show that the decoder can be restricted to a noise-adding mechanism. Interestingly, the RDP function characterized for the case of zero perception loss coincides with that of the marginal metric, and further zero perception loss can be achieved with a 3-dB penalty in minimum distortion. Finally we specialize to the case of Gaussian sources, and derive the RDP function for Gaussian vector case and propose a reverse water-filling type solution. We also partially characterize the RDP function for a mixture of Gaussian vector sources.

In federated learning (FL), the communication constraint between the remote users and the Parameter Server (PS) is a crucial bottleneck. This paper proposes M22, a rate-distortion inspired approach to model update compression for distributed training of deep neural networks (DNNs). In particular, (i) we propose a family of distortion measures referred to as "M-magnitude weighted L2" norm, and (ii) we assume that gradient updates follow an i.i.d. distribution with two degrees of freedom – generalized normal and Weibull distributions. To measure the gradient compression performance under a communication constraint, we define the per-bit accuracy as the optimal improvement in accuracy that a bit of communication brings to the centralized model over the training period. Using this performance measure, we systematically benchmark the choice of gradient distributions and the distortion measure. We provide substantial insights on the role of these choices and argue that significant performance improvements can be attained using such a rate-distortion inspired compressor.

We study causal, low-latency, sequential video compression when the output is subjected to both a mean squared-error (MSE) distortion loss as well as a perception loss to target realism. Motivated by prior approaches, we consider two different perception loss functions (PLFs). The first, PLF-JD, considers the joint distribution (JD) of all the video frames up to the current one, while the second metric, PLF-FMD, considers the framewise marginal distributions (FMD) between the source and reconstruction. Using information theoretic analysis and deep-learning based experiments, we demonstrate that the choice of PLF can have a significant effect on the reconstruction, especially at low-bit rates. In particular, while the reconstruction based on PLF-JD can better preserve the temporal correlation across frames, it also imposes a significant penalty in distortion compared to PLF-FMD and further makes it more difficult to recover from errors made in the earlier output frames. Although the choice of PLF decisively affects reconstruction quality, we also demonstrate that it may not be essential to commit to a particular PLF during encoding and the choice of PLF can be delegated to the decoder. In particular, encoded representations generated by training a system to minimize the MSE (without requiring either PLF) can be {\em near universal} and can generate close to optimal reconstructions for either choice of PLF at the decoder. We validate our results using (one-shot) information-theoretic analysis, detailed study of the rate-distortion-perception tradeoff of the Gauss-Markov source model as well as deep-learning based experiments on moving MNIST and KTH datasets.

Recent years have witnessed significant progress in the area of single image dehazing, thanks to the employment of deep neural networks and diverse datasets. Most of the existing methods perform well when the training and testing are conducted on a single dataset. However, they are not able to handle different types of hazy images using a dehazing model trained on a particular dataset. One possible remedy is to perform training on multiple datasets jointly. However, we observe that this training strategy tends to compromise the model performance on individual datasets. Motivated by this observation, we propose a test-time training method which leverages a helper network to assist the dehazing model in better adapting to a domain of interest. Specifically, during the test time, the helper network evaluates the quality of the dehazing results, then directs the dehazing network to improve the quality by adjusting its parameters via self-supervision. Nevertheless, the inclusion of the helper network does not automatically ensure the desired performance improvement. For this reason, a metalearning approach is employed to make the objectives of the dehazing and helper networks consistent with each other. We demonstrate the effectiveness of the proposed method by providing extensive supporting experiments.

A distributed lossy compression network with$L$encoders and a decoder is considered. Each encoder observes a source and sends a compressed version to the decoder. The decoder produces a joint reconstruction of target signals with the mean squared error distortion below a given threshold. It is assumed that the observed sources can be expressed as the sum of target signals and corruptive noises which are independently generated from two symmetric multivariate Gaussian distributions. The minimum compression rate of this network versus the distortion threshold is referred to as the rate-distortion function, for which an explicit lower bound is established by solving a minimization problem. Our lower bound matches the well-known Berger-Tung upper bound for some values of the distortion threshold. The asymptotic gap between the upper and lower bounds is characterized in the large$L$limit.

In this paper, we consider sequential testing over a single-sensor, a single-decision center setup. At each time, instant t, the sensor gets k samples $(k \gt 0)$ and describes the observed sequence until time t to the decision center over a zero-rate noiseless link. The decision center sends a single bit of feedback to the sensor to request for more samples for compression/testing or to stop the transmission. We have characterized the optimal exponent of type-II error probability under the constraint that type-I error probability does not exceed a given threshold $\varepsilon \in(0,1)$ and also when the expectation of the number of requests from decision center is smaller than n which tends to infinity. Interestingly, the optimal exponent coincides with that for fixed-length hypothesis testing with zero-rate communication constraints.

We consider the fundamental limits of reliable communication over a two-state compound channel when the state of the channel needs to be masked. Our model is closely related to an area of study known as covert communication, a setting in which the transmitter wishes to communicate to legitimate receiver(s) while ensuring that the communication is not detected by an adversary. Our main contribution is the establishment of upper and lower bounds on the throughput-key length region when the constraint that quantifies how much the states are masked is defined to be the total variation distance between the channel output distributions of the two states. When length of the key is sufficiently large, we provide sufficient conditions for the bounds to coincide. Our results follow the so-called square-root law and hence are reminiscent of results in covert communications. Numerical examples, including that of a Gaussian channel, are provided to illustrate our results.

In this paper, the sum secure degrees of freedom (SDoF) of the K-user Multiple Input/Single Output (MISO) Broadcast Channel with Confidential Messages (BCCM) and alternating Channel State Information at the Transmitter (CSIT) is investigated. In the MISO BCCM, a K-antenna transmitter (TX) communicates toward K single-antenna receivers (RXs), so that message for RX k is kept secret from RX j with jsum= (2K − 1)/2. Interestingly, this SDoFsumis attained by a rather simple achievability in which the TX uses artificial noise to prevent the decoding of the message of the unintended receivers at RX 1. The proof for the case K = 3 is discussed in detail.

Coding and testing schemes for binary hypothesis testing over noisy networks are proposed and their corresponding type-II error exponents are derived. When communication is over a discrete memoryless channel (DMC), our scheme combines Shimokawa-Han-Amari's hypothesis testing scheme with Borade-Nakiboglu-Zheng's unequal error protection (UEP) for channel coding where source and channel codewords are simultaneously decoded. The resulting exponent is optimal for the newly introduced class of generalized testing against conditional independence. When communication is over a multi-access channel (MAC), our scheme combines hybrid coding with UEP. The resulting error exponent over the MAC is optimal in the case of generalized testing against conditional independence with independent observations at the two sensors when the MAC decomposes into two individual DMCs. In this case, separate source-channel coding is sufficient and no UEP is required. This same conclusion holds also under arbitrarily correlated sensor observations when testing is against independence.

In this paper, we study covert communication over a compound discrete memoryless channel (DMC). There are two channel states in which one of them is arbitrarily chosen and remains fixed during the transmission. The objective is to reliably send a message from the transmitter to the receiver. An adversary who is observing the channel output should not be able to infer the channel state. Two covertness metrics are considered. In the first metric, covertness is measured using the KL-divergence of the channel output marginal of each state with a fixed distribution. Different cases where such a distribution can be specified, are studied. The optimal transmission rate of each case is established. In the second metric, the covertness is measured by using the total variation distance of the channel output marginals of the two states. Upper and lower bounds on the optimal transmission covert rate are derived. The bounds match for a special case and characterize the optimal throughput.

Coding and testing schemes and the corresponding achievable type-II error exponents are presented for binary hypothesis testing over two-hop relay networks. The schemes are based on cascade source coding techniques and unanimous decision-forwarding, the latter meaning that a terminal decides on the null hypothesis only if all previous terminals have decided on the null hypothesis. If the observations at the transmitter, the relay, and the receiver form a Markov chain in this order, then, without loss in performance, the proposed cascade source code can be replaced by two independent point-to-point source codes, one for each hop. The decoupled scheme (combined with decision-forwarding) is shown to attain the optimal type-II error exponents for various instances of “testing against conditional independence.” The same decoupling is shown to be optimal also for some instances of “testing against independence,” when the observations at the transmitter, the receiver, and the relay form a Markov chain in this order and when the relay-to-receiver link is of sufficiently high rate. For completeness, this paper also presents an analysis of the Shimokawa-Han-Amari binning scheme for the point-to-point hypothesis testing setup.

Consider distributed hypothesis testing over multiple-access channels (MACs), where the receiver wishes to maximize the type-II error exponent under a constrained type-I error probability. For this setup, we propose a scheme that combines hybrid coding with a MAC-version of Borades unequal error protection. It achieves the optimal type-II error exponent for a generalization of testing against independence over an orthogonal MAC when the transmitters' sources are independent. In this case, hybrid coding can be replaced by the simpler separate source-channel coding. The paper also presents upper and lower bounds on the optimal type-II error exponent for generalized testing against independence of Gaussian sources over a Gaussian MAC. The bounds are close and significantly larger than a type-II error exponent that is achievable using separate source-channel coding.

We revisit the hypothesis testing with communication constraints problem, also called distributed hypothesis testing, from the viewpoint of privacy. Instead of observing the raw data directly, the transmitter observes a sanitized or randomized version of it. We impose an upper bound on the mutual information between the raw and randomized data. Under this scenario, the decoder, which is also provided with side information, is required to make a decision on whether the null or alternative hypothesis is in effect. First, we provide a general lower bound on the type-II exponent for arbitrary hypotheses, privacy mechanism, rates, and leakage parameters. Second, we consider the testing against independence scenario in which the distribution under the alternative hypothesis is the product of the marginals of the distribution under the null hypothesis. In this setup, we show that the exponent is known exactly and the strong converse property holds. Finally, the trade-offs between the exponent, compression rate, and leakage parameter are illustrated through a binary example.

A distributed binary hypothesis testing problem is studied with one observer and two decision centers. Achievable type-II error exponents are derived for testing against conditional independence when the observer communicates with the two decision centers over one common and two individual noise-free bit pipes and when it communicates with them over a noisy broadcast channel. The results are based on a coding and testing scheme that splits the observations into subblocks, so that transmitter and receivers can independently apply to each subblock either Gray-Wyner coordination coding with side-information or hybrid joint source-channel coding with side-information, followed by a Neyman-Pearson test over the subblocks at the receivers. This approach allows to avoid introducing further error exponents that one would expect from the receivers' decoding operations related to binning or the noisy transmission channel. The derived exponents are shown to be optimal in some special cases when communication is over noise-free links. The results reveal a tradeoff between the type-II error exponents at the two decision centers.

Binary hypothesis testing over single and parallel cascade channels is considered where sensors communicate with dedicated relays, and these relays with a single final receiver. All relays as well as the final receiver decide on the binary hypothesis governing the joint probability distribution of the observations at the sensors, relays, and final receiver. The quantity of interest is the set of feasible type-II error exponents that allow for the type-I error probabilities to vanish asymptotically as the observation length increases. A coding scheme is proposed and the corresponding set of feasible type-II error exponents is analyzed by means of a modified Han-type analysis that can account for distributed decisions based on different codebooks and for nodes forwarding their decisions to other nodes. The obtained exponent region is optimal in some special cases.

In this study, the authors study lossy communication of correlated sources over a multiple‐access wiretap channel (MAC‐WT). Consider a system with two transmitters, a receiver and an eavesdropper. There are two correlated sources where each of them is observed by the corresponding transmitter, separately. Each transmitter wishes to describe its source sequence to the receiver with a desired distortion. The sources need to be kept secret from the eavesdropper. They find an achievable region for the MAC‐WT with correlated sources by separation. A joint source‐channel coding scheme for the MAC‐WT is also proposed. They consider lossy communication of a bivariate Gaussian source over Gaussian MAC‐WT (GMAC‐WT). They propose a separation‐based achievable scheme for the GMAC‐WT. An achievable region for the GMAC‐WT based on uncoded transmission is also found. They compare the separate and the uncoded schemes for the symmetric GMAC‐WT, where the same constraint on the power of each transmitter is imposed and the same distortion on each source is achieved. For another case of source correlation coefficient, they compare the separate, uncoded and hybrid schemes. They obtain outer bounds to the rate‐distortion‐equivocation region of: (i) the degraded MAC‐WT where the output at the eavesdropper is a degraded version of the output at the receiver, (ii) the GMAC‐WT with independent sources and (iii) the symmetric GMAC‐WT when the correlation of the sources is maximum. Optimal regions for some cases are established.

In this study, the authors consider the secrecy of a one‐receiver, two‐eavesdropper broadcast channel (BC) with three degraded message sets. Consider a three‐receiver BC with three messages, where the first message is decoded by all the receivers. The second message is decoded by the first and the second receivers and is to be kept secret from the third receiver. The third message is decoded by the first receiver and is to be kept secret from the second and the third receivers. The authors consider the imperfect secrecy condition at the second receiver, that is, it is allowed to partially decode the third message. However, the perfect secrecy condition at the third receiver, does not allow it to decode the confidential messages. The coding scheme for this model requires decoding strategy for finding the messages at different destinations. The authors propose a coding scheme which uses indirect decoding. The authors also obtain an outer bound and use it to determine the secrecy capacity region of some classes of one‐receiver, two‐eavesdropper BCs with three degraded message sets. The authors extend our results to the Gaussian case and evaluate the achievable region.

In this paper, we consider lossy source transmission over a Multiple Access WireTap Channel (MAC-WT). In this model, there are two correlated sources, each of them is available at the corresponding encoder. The receiver tries to reconstruct both sources with desired distortions. The sources should be kept secret from an eavesdropper. We propose a joint source-channel scheme for the MAC-WT. In this scheme, each source sequence is mapped to common and private codewords. The common codeword can be decoded by the eavesdropper. The private codeword needs to be kept secret from the eavesdropper by using Wyner's wiretap coding. We also discuss some special cases of the proposed scheme.

In this paper, we study the one-receiver two-eavesdropper Broadcast Channel (BC) with three degraded message sets. A common message is sent to three receivers. Another message is sent to the first and second receivers and needs to be kept secret from the third receiver (second eavesdropper). The third message is sent to the first receiver and needs to be kept secret from the second and third receivers (first and second eavesdroppers). First, we consider perfect secrecy conditions at the eavesdroppers, where we find an achievable perfect secrecy region. In the achievability scheme, we use superposition coding which divides the available randomness into different levels. These levels are used to mislead the eavesdroppers. The proposed coding scheme determines the perfect secrecy capacity region of the one-receiver two-eavesdropper BC with three degraded message sets, when the receivers exhibit a degradedness order. Next, we find an achievable rate-equivocation region using the rate splitting technique and indirect decoding. We show that for the two-receiver one-eavesdropper BC with three degraded message sets where we relax the secrecy condition at the second receiver, our coding scheme is optimal when the second receiver is a degraded version of the first receiver and it is less noisy than the third receiver.

We investigate the secrecy capacity region of 2-receiver, 1-eavesdropper Broadcast Channel (BC) with two causal states and conferencing decoders. The encoder sends two messages, one of them for both legitimate receivers and the other one for the first legitimate receiver. It keeps these messages secret from the eavesdropper. Each state is causally available at the corresponding receiver, while both states are known causally at the encoder. Each receiver wishes to send the state which is not available at the other receiver, through a noiseless link with limited capacity. We find an inner bound on the secrecy capacity region of this channel. The achievability scheme employs block Markov coding and Shannon strategy for channels with state. In each block, we make use of the states to generate two keys which will be used in the next block. The idea is to utilize the cooperation of both receivers to recover both keys at each receiver while ensuring secrecy. We also establish an outer bound on the secrecy capacity region of the 2-rec, 1-eav. BC with Causal Channel State Information (CCSI) and conferencing. We prove that the proposed bounds coincide for a special case.

We study the reliable transmission of correlated sources over the relay channel. One of the sources is available at the sender while the other one is known to the relay. The receiver wishes to reconstruct both sources. We find necessary and sufficient conditions for optimal separation of source and channel codes. The coding scheme is based on the combination of the Slepian-Wolf source coding and Partial Decode-and-Forward (PDF) strategy. In this scheme, the relay partially decodes the message transmitted by the sender. We also consider the semi-deterministic relay channel where the output at the relay is a deterministic function of inputs from the sender and the relay. For the transmission of a single source over the semi-deterministic relay channel with correlated side information at the relay, the proposed conditions coincide. We also find necessary and sufficient conditions for the transmission of sources over the degraded relay channel where the output at the receiver is a degraded version of the output at the relay.

In this paper, we consider the 3-receiver broadcast channel with one common and two confidential messages. One of the confidential messages is sent to the first and second receivers and kept secret from the third receiver (eavesdropper). The other confidential message is sent to the first receiver and kept secret from the second and third receivers (eavesdroppers). The common message is decoded by all receivers. We study some special cases of the model and determine their secrecy capacity regions.

In this paper, we consider the reliable transmission of correlated sources over the relay channel. The destination wishes to find a lossless reconstruction of both sources. We find the sufficient conditions under which the relay can partially cooperate with the sender to transmit the sources to the destination. The achievability scheme is based on regular encoding/backward decoding strategy. The proposed region includes the known region for multiple-access channel with correlated sources and the achievable rate of relay channel with partial decode-and-forward strategy, as special cases. A class of deterministic relay channels is considered. It is shown that the error-free transmission of correlated sources is possible over this channel, using the proposed joint source-channel coding scheme.

In this paper, we obtain an outer bound on the capacity region of the Broadcast-Relay-Channel (BRC). BRC is a four node network with one source, one relay and two destinations. To the best of our knowledge, this work represents the first effort to find an outer bound on the capacity region of the BRC. In the proof, we jointly utilize the ideas of the outer bounds on the Broadcast Channel (BC) and also the cut-set bound. The proposed outer bound looks like the inner bound obtained by Kramer et. al for BRC.

In this paper, we consider Z Channel (ZC) with cooperation, where there are two senders and two receivers. The first sender transmits information to both receivers and also to the second sender. The first receiver and the second sender cooperate with the first sender in sending messages to the second receiver. The second sender transmits information only to its intended receiver. We first establish an achievable rate region for the general cooperative ZC. The coding scheme is based on rate splitting technique. At the receivers, we use joint decoding strategy. To illustrate the tightness of the proposed inner bound, we define degraded cooperative ZC. We derive an outer bound to the capacity region of the degraded cooperative ZC. It will be shown that the two bounds coincide and determine the capacity region of the degraded cooperative ZC.

In this paper, we consider the two-user Z Channel (ZC). There are two senders and two receivers in the model. One of the senders transmits information to its intended receiver, while the other sender transmits information to both receivers. We establish a new achievable rate region for the ZC, using Marton's binning technique. At the receivers, we use joint decoding strategy. To illustrate the tightness of the proposed inner bound, we consider the degraded ZC which was first introduced by Chong et. al. We then characterize the capacity region of this type of degraded ZC.

The authors consider the problem of broadcast-relay-networks with two cooperative relays. There are a transmitter, two relays and two receivers in the network. The message of the transmitter intended to the receivers has common and private parts. The relays fully cooperate with each other and with the transmitter to send the common part of the message, whereas the private parts of the message are sent through the direct links between the transmitter and receivers. The authors found an achievable rate region for this network by using the symmetric relaying strategy. In this strategy, each relay completely decodes the message of other relay. In the proof the authors took advantage of regular encoding/sliding window decoding at relays and simultaneous backward decoding analysis at receivers. Marton's broadcast code construction is used at the transmitter to split the rate between the users. Three special cases of achievable rate region are shown: (i) Kramer's achievable rate region for broadcast relay channel; (ii) Ghabeli's achievable rate for symmetric two-relay network; and (iii) Marton's achievable rate region for broadcast channel with common message. The additive white Gaussian noise model is also considered and the achievable rate region of Gaussian network is discussed.

The authors introduce a generalisation of the multiple-access-relay-channel (MARC) called multiple-access-relay-network (MARN). The MARC model was first introduced by Kramer et al. and consists of many transmitters, one receiver and only one relay. In the MARN, there are many transmitters, many relays and one receiver. The authors obtained an achievable rate region for MARN by considering partial decode-and-forward (PDF) strategy at the relays. They showed that the region obtained using PDF strategy subsumes the region obtained by Kramer et al. for MARC. In the proposed coding scheme, the authors take advantage of PDF strategy based on regular encoding/backward decoding strategy. They also define semi-deterministic MARN, in which the output of the link between each transmitter and each relay is a deterministic function of the transmitter's input. The authors obtained an inner bound (using our achievable rate region) and also an outer bound on the capacity region of the semi-deterministic MARN and compared them.