Transformer-based neural decoders have emerged as a promising approach to error correction coding, combining data-driven adaptability with efficient modeling of long-range dependencies. This paper presents a novel decoder architecture that integrates classical belief propagation principles with transformer designs. We introduce a differentiable syndrome loss function leveraging global codebook structure and a differential-attention mechanism optimizing bit and syndrome embedding interactions. Experimental results demonstrate consistent performance improvements over existing transformer-based decoders, with our approach surpassing traditional belief propagation decoders for short-to-medium length LDPC codes.

Ruzsa’s equivalence theorem provided a framework for converting certain families of inequalities in additive combinatorics to entropic inequalities (which sometimes did not possess stand-alone entropic proofs). In this work, we first establish formal equivalences between some families (different from Ruzsa) of inequalities in additive combinatorics and entropic ones. As a first step to further these equivalences, we establish an information-theoretic characterization of the magnification ratio that could also be of independent interest.

The doubling-followed-by-rotation trick to prove the extremality of Gaussian distributions has been a valuable tool in information theory. In particular, the above trick has been used to establish the Gaussian extremality of a family of inequalities that unifies the Entropy Power Inequality and the Brascamp-Lieb inequalities. Here, we develop a technique (similar to the one in the continuous case) to prove the extremality of Haar distributions for a similar family of inequalities in finite Abelian groups.

Ruzsa’s equivalence theorem provided a framework for converting certain families of inequalities in additive combinatorics to entropic inequalities (which sometimes did not possess stand-alone entropic proofs). In this work, we first establish formal equivalences between some families (different from Ruzsa) of inequalities in additive combinatorics and entropic ones. Secondly, we provide stand-alone entropic proofs for some previously known entropic inequalities that we established via Ruzsa’s equivalence theorem. As a first step to further these equivalences, we provide an information theoretic characterization of the magnification ratio that is also of independent interest.

In this paper we derive an inequality relating linear combinations of mutual information between subsets of mutually independent random variables and an auxiliary random variable. One choice of a family of auxiliary variables leads to a new proof of a Stam-type inequality regarding the Fisher Information of sums of independent random variables. Another choice of a family of auxiliary random variables leads to new results as well as new proofs of results relating to strong data processing constants and maximal correlation between sums of independent random variables. Other results obtained include convexity of Kullback–Leibler divergence over a parameterized path along pairs of binomial and Poisson distributions, as well as a new duality-based argument relating the Stam-type inequality and entropy power inequality.

In this paper we derive an inequality relating linear combinations of mutual information between subsets of mutually independent random variables and an auxiliary random variable. As corollaries of this inequality, we obtain new results and generalizations and new proofs of known results.

Certain families of non-convex optimization problems involving linear combinations of log-determinants of positive definite matrices are shown to have a unique local maximizer. These geometric results are established using information-theoretic arguments. We demonstrate these results for three different families: two of which arise in the study of the capacity region of the vector Gaussian broadcast channel and another one in the study of computing the optimal generalized Brascamp-Lieb constant.

We generalize a convexity result due to Wyner and Ziv to channels with binary inputs and arbitrary outputs. This results in a convex reformulation of some non-convex optimization problems that arise naturally in multi-user information theory.