We extend the results of Ghourchian et al. [1] to joint source-channel coding with eavesdropping. Our work characterizes the sequential encoding process using the cumulative rate distribution functions (CRDF) and includes a security constraint using the cumulative leakage distribution functions (CLF). The information leakage is defined based on the mutual information between the source and the output of the wiretap channel to the eavesdropper. We derive inner and outer bounds on the achievable CRDF for a given source and CLF, and show that the bounds are tight when the distribution achieving the capacity of the wiretap channel is the same as the one achieving the capacity of the channel.

We study the problem of rate-distortion equivocation with side-information only available at the decoder when an independent private random key is shared between the sender and the receiver. The sender compresses the sequence, and the receiver reconstructs it such that the average distortion between the source and the output is limited. The equivocation is measured at an eavesdropper that intercepts the source encoded message, utilizing side-information correlated with the source and the side-information at the decoder. We have derived the entire achievable rate-distortion-equivocation region for this problem.

We introduce the concept of achievable cumulative rate distribution functions (CRDF) to characterize sequentially encoding processes that ensure a lossless or lossy reconstruction subject to an average distortion using a non-causal decoder. Utilizing tools from majorization theory, we derive necessary and sufficient conditions on the CRDF for a given IID source. It turns out that the optimal achievable distortion level can be adequately characterized by the concave-hull of the CRDF.

The sparsity and compressibility of finite-dimensional signals are of great interest in fields, such as compressed sensing. The notion of compressibility is also extended to infinite sequences of independent identically distributed or ergodic random variables based on the observed error in their nonlinear k-term approximation. In this paper, we use the entropy measure to study the compressibility of continuous-domain innovation processes (alternatively known as white noise). Specifically, we define such a measure as the entropy limit of the doubly quantized (time and amplitude) process. This provides a tool to compare the compressibility of various innovation processes. It also allows us to identify an analogue of the concept of “entropy dimension" which was originally defined by Rényi for random variables. Particular attention is given to stable and impulsive Poisson innovation processes. Here, our results recognize Poisson innovations as the more compressible ones with an entropy measure far below that of stable innovations. While this result departs from the previous knowledge regarding the compressibility of impulsive Poisson laws compared with continuous fat-tailed distributions, our entropy measure ranks α-stable innovations according to their tail.

In some applications, the variance of additive measurement noise depends on the signal that we aim to measure. For instance, additive signal-dependent Gaussian noise (ASDGN) channel models are used in molecular and optical communication. Herein, we provide lower and upper bounds on the capacity of additive signal-dependent noise (ASDN) channels. The first lower bound is based on an extension of majorization inequalities, and the second lower bound utilizes the properties of the differential entropy. The lower bounds are valid for arbitrary ASDN channels. The upper bound is based on a previous idea of the authors (“symmetric relative entropy”) and is applied to the ASDGN channels. These bounds indicate that in the ASDN channels (unlike the classical additive white Gaussian noise channels), the capacity does not necessarily become larger by reducing the noise variance function. We also provide sufficient conditions under which the capacity becomes infinite. This is complemented by some conditions implying that the capacity is finite, and a unique capacity achieving measure exists (in the sense of the output measure).