Federated learning (FL) is a widely used and impactful distributed optimization framework that achieves consensus by averaging locally trained models. While effective, this approach may not align well with Bayesian inference, where the model space is more naturally represented as a distribution space. Taking an information-geometric perspective, we reinterpret FL aggregation as the problem of finding the barycenter of local posteriors using a predefined divergence metric, minimizing the average discrepancy across clients. This perspective provides a unifying framework that generalizes many existing methods and offers crisp insights into their theoretical underpinnings. We then propose BA-BFL, an algorithm that retains the convergence properties of Federated Averaging in non-convex settings. In non-independent and identically distributed scenarios, we conduct extensive comparisons with statistical aggregation techniques, showing that BA-BFL achieves performance comparable to state-of-the-art methods while also providing a geometric interpretation of the aggregation phase. Additionally, we extend our analysis to Hybrid Bayesian Deep Learning, exploring the impact of Bayesian layers on uncertainty quantification and model calibration.

In this paper, we investigate the rate-distortion-perception function (RDPF) of a source modeled as a Gaussian Process (GP) over a measure space$\Omega$, under mean squared error (MSE) distortion and squared Wasserstein-2 perception metrics. First, we show that the optimal reconstruction process is itself a GP, whose covariance operator shares the same set of eigenvectors as the source's covariance operator. This structural property, akin to the classical rate-distortion function (RDF), allows us to reformulate the RDPF problem in terms of the Karhunen-Loève (KL) transform coefficients of the involved GPs. Leveraging the similarities with the finite-dimensional Gaussian RDPF, we derive a tight analytical upper bound on the RDPF for GPs, which recovers the optimal solution in the “perfect realism” regime. Finally, for stationary GPs over the interval$[0, T]$with Lebesgue measure, we derive an upper bound on the rate and distortion for a fixed perceptual level and$T \rightarrow \infty$as a function of the spectral density of the source process. We complement our theoretical findings with relevant simulation studies.

In this paper, we investigate the problem of distributionally robust source coding, i.e., source coding under uncertainty in the source distribution, discussing both the coding and computational aspects of the problem. We propose two extensions of the so-called Strong Functional Representation Lemma (SFRL), considering the cases where, for a fixed conditional distribution, the marginal inducing the joint coupling belongs to either a finite set of distributions or a Kullback-Leibler divergence sphere (KL-Sphere) centered at a fixed nominal distribution. Using these extensions, we derive distributionally robust coding schemes for both the one-shot and asymptotic regimes, generalizing previous results in the literature. Focusing on the case where the source distribution belongs to a given KL-Sphere, we derive an implicit characterization of the points attaining the robust rate-distortion function (R-RDF), which we later exploit to implement a novel algorithm for computing the R-RDF. Finally, we characterize the analytical expression of the R-RDF for Bernoulli sources, providing a theoretical benchmark to evaluate the estimation performance of the proposed algorithm.

We study the computation of the rate-distortion-perception function (RDPF) for discrete memoryless sources subject to a single-letter average distortion constraint and a perception constraint belonging to the family off-divergences. In this setting, the RDPF forms a convex programming problem for which we characterize optimal parametric solutions. We employ the developed solutions in an alternating minimization scheme, namely Optimal Alternating Minimization (OAM), for which we provide convergence guarantees. Nevertheless, the OAM scheme does not lead to a direct implementation of a generalized Blahut-Arimoto (BA) type of algorithm due to implicit equations in the iteration’s structure. To overcome this difficulty, we propose two alternative minimization approaches whose applicability depends on the smoothness of the used perception metric: a Newton-based Alternating Minimization (NAM) scheme, relying on Newton’s root-finding method for the approximation of the optimal solution of the iteration, and a Relaxed Alternating Minimization (RAM) scheme, based on relaxing the OAM iterates. We show, by deriving necessary and sufficient conditions, that both schemes guarantee convergence to a globally optimal solution. We also provide sufficient conditions on the distortion and perception constraints, which guarantee that the proposed algorithms converge exponentially fast in the number of iteration steps. We corroborate our theoretical results with numerical simulations and establish connections with existing results.

In this paper, we propose a generic method for computing the rate-distortion-perception function (RDPF) of a multivariate Gaussian source under tensorizable distortion and perception metrics. Through the assumption of a jointly Gaussian reconstruction, we establish that the optimal solution of the RDPF belongs to the vector space spanned by the eigenvector of the source covariance matrix. Consequently, the multivariate optimization problem can be expressed as a function of the scalar Gaussian RDPFs of the source marginals, constrained by global distortion and perception levels. Utilizing this result, we devise an alternating minimization scheme based on the block nonlinear Gauss-Seidel method. This scheme solves optimally the optimization problem while identifying the optimal stage-wise distortion and perception levels. Furthermore, the associated algorithmic embodiment is provided, along with the convergence and the rate of convergence characterization. Lastly, in the regime of “perfect realism”, we provide the analytical solution for the multivariate Gaussian RDPF. We corroborate our findings with numerical simulations and draw connections to existing results.

We present a new method to estimate the rate-distortion-perception function in the perfect realism regime (PR-RDPF), for multivariate continuous sources subject to a single-letter average distortion constraint. The proposed approach is not only able to solve the specific problem but also two related problems: the entropic optimal transport (EOT) and the output-constrained rate-distortion function (OC-RDF), of which the PR-RDPF represents a special case. Using copula distributions, we show that the OC-RDF can be cast as an$I$-projection problem on a convex set, based on which we develop a parametric solution of the optimal projection proving that its parameters can be estimated, up to an arbitrary precision, via the solution of a convex program. Subsequently, we propose an iterative scheme via gradient methods to estimate the convex program. Lastly, we characterize a Shannon lower bound (SLB) for the PR-RDPF under a mean squared error (MSE) distortion constraint. We support our theoretical findings with numerical examples by assessing the estimation performance of our iterative scheme using the PR-RDPF with the obtained SLB for various sources.

In this paper, we study the computation of the rate-distortion-perception function (RDPF) for discrete memoryless sources subject to a single-letter average distortion constraint and a perception constraint that belongs to the family of f-divergences. For that, we leverage the fact that RDPF, assuming mild regularity conditions on the perception constraint, forms a convex programming problem. We first develop parametric characterizations of the optimal solution and utilize them in an alternating minimization approach for which we prove convergence guarantees. The resulting structure of the iterations of the alternating minimization approach renders the implementation of a generalized Blahut-Arimoto (BA) type of algorithm infeasible. To overcome this difficulty, we propose a relaxed formulation of the structure of the iterations in the alternating minimization approach, which allows for the implementation of an approximate iterative scheme. This approximation is shown, via the derivation of necessary and sufficient conditions, to guarantee convergence to a globally optimal solution. We also provide sufficient conditions on the distortion and the perception constraints which guarantee that our algorithm converges exponentially fast. We corroborate our theoretical results with numerical simulations, and we draw connections with existing results.

Synthetic DNA has received much attention recently as a long-term archival medium alternative due to its high density and durability characteristics. However, most current work has primarily focused on using DNA as a precise storage medium. In this work, we take an alternate view of DNA. Using neural-network-based compression techniques, we transform images into a latent-space representation, which we then store on DNA. By doing so, we transform DNA into an approximate image storage medium, as images generated back from DNA are only approximate representations of the original images. Using several datasets, we investigate the storage benefits of approximation, and study the impact of DNA storage errors (substitutions, indels, bias) on the quality of approximation. In doing so, we demonstrate the feasibility and potential of viewing DNA as an approximate storage medium.