We focus on a (generic) joint source-channel coding problem, appearing in a broad variety of real-world application. Explicitly, a noisy observation from a user/source signal should be compressed, ahead of getting forwarded over an error-prone and rate-limited channel to a remote processing unit. The design problem shall be formulated in a fashion that the impacts of the forward link are taken into account. Aligned with the Information Bottleneck (IB) method, we consider the Mutual Information (MI) as the fidelity criterion, and work out a data-driven approach to tackle the underlying design problem based upon a finite sample set. For that, we derive a tractable variational lower-bound of the objective functional, and present a general learning architecture which can be used to optimize the given lower-bound by standard training of the encoder and decoder Deep Neural Networks. This approach that is, principally, based upon the (generative) latent variable models, extends the concepts of Variational AutoEncoder (VAE) and Deep Variational Information Bottleneck (Deep VIB) for (remote) source coding to the context of joint source-channel coding. We validate the effectiveness of our approach by several numerical simulations over typical transmission scenarios.

Consider the following setup: Through ajointdesign, multiple observations of a remote data source shall belocallycompressed before getting transmitted via severalerror-prone, rate-limited forward links to a (distant) processing unit. For addressing this specific instance of multiterminalJoint Source-Channel Codingproblem, in this article, the foundational principle of theInformation Bottleneckmethod is fully extended to obtain purely statistical design approaches, enjoying theMutual Informationas their fidelity criterion. Specifically, the forms of stationary points for two types of distributed compression schemes are characterized here. Subsequently, those acquired solutions are utilized as the centerpiece of the proposed generic, iterative algorithm, termed theMultiterminal Forward-Aware Vector Information Bottleneck (M-FAVIB), for addressing the design optimizations. Leveraging an unfolding trick, it will be proven that both distributed compression schemes fall into the category ofSuccessive Upper-Bound Minimization, ensuring their convergence to a stationary point. Eventually, the effectiveness of the proposed compression schemes will be substantiated as well by means of numerical investigations over some typical transmission scenarios.

The focus is on Wyner-Ziv type distributed fronthaul compression for the uplink of Cloud Radio Access Networks with single-hop topology to leverage the correlation among the received signals of neighboring Radio Access Points. For this, we highlight the relation between the problem at hand and the Chief Executive Officer source coding under logarithmic-loss distortion and depict that the achievability arguments from the latter verify addressing the postulated optimization. Subsequently, we derive the pertinent optimal solution and utilize that as the backbone of the Generalized Distributed Information Bottleneck (G-DIB) routine proposed here to tackle the considered remote source coding problem. As its name suggests, this novel approach in its very core spirit extends the State-of-the-Art Distributed Information Bottleneck (DIB) method by enabling individual rate constraints for various fronthaul links.

The main focus will be on the indirect Joint Source-Channel Coding problem in which a noisy observation of the source has to be quantized ahead of transmission over an error-prone forward link to a remote processing unit. To that end, we present here a complete extension to the preliminary Information Bottleneck method by providing the formal optimal solution to this newly established Variational Principle, together with an algorithm, the Forward-Aware Vector Information Bottleneck (FAVIB), to pragmatically tackle its underlying non-convex design optimization. FAVIB extends the current state-of-the-art approaches via capacitating a full sweep over the entire gamut of the trade-off parameter. Consequently, the trajectory of all achievable points in the Information-Compression plane becomes traversable via soft mappings. It will be shown that, by enjoying an inherent error protection, this novel compression scheme can obviate the call for separate channel coding on the forward path.

Consider following setup: A number of observations from a data source shall be compressed jointly prior to a forward transmission via several rate- limited links to a central processing unit. To design the respective quantizers, here, Mutual Information is chosen as the fidelity criterion and the broad-ranging structure of Multivariate Information Bottleneck is then aptly tailored to that purpose. This, indeed, not only yields a novel design approach for the considered distributed scenario but also paves the way towards perceiving the chance of leveraging this flexible conceptual frame in a vast variety of applications regarding digital data transmission. Explicitly, it immediately enables addressing various extensions of the presumed arrangement, incorporating the parallel construction of intertwined compression systems for several correlated sources.

The main focus of this paper is on the problem of noisy source coding wherein observed signals from an inaccessible source shall be compressed. To that end, rather than resorting to the conventional methods from Rate-Distortion theory, the so-called Information Bottleneck paradigm is deployed in order to obtain a highly informative representing signal w.r.t. the given source. An efficient, generic and highly flexible graph-based message passing routine for clustering, known as the Affinity Propagation is successfully applied here as a novel treatment for that purpose. The fundamental differences and the performance-wise comparison w.r.t. the state-of-the-art KL-Means-IB algorithm is provided as well.

In the context of noisy source coding, contrary to the conventional Rate-Distortion theory, the so-called Information Bottleneck method formulates the existent fundamental complexity-precision trade-off in a symmetric and purely information-theoretic fashion. Since the pertinent optimization task to design the quantizer is quite demanding, a number of heuristics have been developed to provide practically feasible procedures at the expense of yielding suboptimal solutions. In this paper, we consider two pertinent routines originally appeared in totally different applications and set out to precisely prove their algorithmic equivalence by conducting a thorough analysis over the corresponding algorithmic steps. We further corroborate our theoretical investigation employing computer-based simulations.

The general problem of quantizing observation signals appears in different aspects of data processing, from special code designs to realization of low-complexity receivers. To this end, a new framework, known as the Information Bottleneck method, has recently attracted a great deal of attention. In this paper, after introducing this framework and providing the Iterative Information Bottleneck algorithm as the primary pertinent solution, we also discuss three other heuristics aiming to solve the similar problem efficiently. Since the resultant solution of considered approaches is locally optimum, it strongly depends on the choice of initialization. The main contribution of this work is to prove the equivalence of these algorithms asymptotically, i.e., assuming an infinite run of algorithms for the extreme case of infinitely large trade-off parameter. We also substantiate this claim by means of computer-based simulations.