The inference of a large symmetric signal-matrix$\boldsymbol{S} \in \mathbb {R}^{N\times N}$corrupted by additive Gaussian noise, is considered for two regimes of growth of the rank M as a function of N. For sub-linear ranks$M=\Theta (N^{\alpha })$with$\alpha \in (0,1)$the mutual information and minimum mean-square error (MMSE) are derived for two classes of signal-matrices: (a)$\boldsymbol{S} =\boldsymbol{X} \boldsymbol{X} ^{\intercal } $with entries of$\boldsymbol{X} \in \mathbb {R}^{N\times M}$independent identically distributed; (b)$\boldsymbol{S} $sampled from a rotationally invariant distribution. Surprisingly, the formulas match the rank-one case. Two efficient algorithms are explored and conjectured to saturate the MMSE when no statistical-to-computational gap is present: (1) Decimation Approximate Message Passing; (2) a spectral algorithm based on a Rotation Invariant Estimator. For linear ranks$M=\Theta (N)$the mutual information is rigorously derived for signal-matrices from a rotationally invariant distribution. Close connections with scalar inference in free probability are uncovered, which allow to deduce a simple formula for the MMSE as an integral involving the limiting spectral measure of the data matrix only. An interesting issue is whether the known information theoretic phase transitions for rank-one, and hence also sub-linear-rank, still persist in linear-rank. Our analysis suggests that only a smoothed-out trace of the transitions persists. Furthermore, the change of behavior between low and truly high-rank regimes only happens at the linear scale$\alpha = 1$.

We propose a rectangular rotational invariant estimator to recover a real matrix from noisy matrix observations coming from an arbitrary additive rotational invariant perturbation, in the large dimension limit. Using the Bayes-optimality of this estimator, we derive the asymptotic minimum mean squared error (MMSE). For the particular case of Gaussian noise, we find an explicit expression for the MMSE in terms of the limiting singular value distribution of the observation matrix. Moreover, we prove a formula linking the asymptotic mutual information and the limit of log-spherical integral of rectangular matrices. We also provide numerical checks for our results, which match our theoretical predictions and known Bayesian inference results.

We consider a statistical model for matrix factorization in a regime where the rank of the two hidden matrix factors grows linearly with their dimension and their product is corrupted by additive noise. Despite various approaches, statistical and algorithmic limits of such problems have remained elusive. We study a Bayesian setting with the assumptions that (a) one of the matrix factors is symmetric, (b) both factors as well as the additive noise have rotational invariant priors, (c) the priors are known to the statistician. We derive analytical formulas for Rotation Invariant Estimators to reconstruct the two matrix factors, and conjecture that these are optimal in the large-dimension limit, in the sense that they minimize the average mean-square-error. We provide numerical checks which confirm the optimality conjecture when confronted to Oracle Estimators which are optimal by definition, but involve the ground-truth. Our derivation relies on a combination of tools, namely random matrix theory transforms, spherical integral formulas, and the replica method from statistical mechanics.

We consider the estimation of a n×m matrix u∗v∗Tobserved through an additive Gaussian noise channel, a problem that frequently arises in statistics and machine learning. We investigate a scenario involving mismatched Bayesian inference in which the statistician is unaware of true prior and uses an assumed prior. We derive the exact analytic expression for the asymptotic mean squared error (MSE) in the large system size limit for the particular case of Gaussian priors and additive noise. Our formulas demonstrate that in the mismatched case, estimation is still possible. Additionally, the minimum MSE (MMSE) can be obtained by selecting a non-trivial set of parameters beyond the matched parameters. Our technique is based on the asymptotic behavior of spherical integrals for rectangular matrices. Our method can be extended to non-rotation-invariant distributions for the true prior but requires rotation invariance for the statistician’s assumed prior.