Unlike the classical linear model, nonlinear generative models have been addressed sparsely in the literature of statistical learning. This work aims to shed light on these models and their secrecy potential. To this end, we invoke the replica method to derive the asymptotic normalized cross entropy in an inverse probability problem whose generative model is described by a Gaussian random field with a generic covariance function. Our derivations further demonstrate the asymptotic statistical decoupling of the Bayesian estimator and specify the decoupled setting for a given nonlinear model. The replica solution depicts that strictly nonlinear models establish an all-or-nothing phase transition: there exists a critical load at which the optimal Bayesian inference changes from perfect to an uncorrelated learning. Based on this finding, we design a new secure coding scheme which achieves the secrecy capacity of the wiretap channel. This interesting result implies that strictly nonlinear generative models are perfectly secured without any secure coding. We justify this latter statement through the analysis of an illustrative model for perfectly secure and reliable inference.

Inverse probability problems whose generative models are given by strictly nonlinear Gaussian random fields show the all-or-nothing behavior: There exists a critical rate at which Bayesian inference exhibits a phase transition. Below this rate, the optimal Bayesian estimator recovers the data perfectly, and above it the recovered data becomes uncorrelated. This study uses the replica method from the theory of spin glasses to show that this critical rate is the channel capacity. This interesting finding has a particular application to the problem of secure transmission: A strictly nonlinear Gaussian random field along with random binning can be used to securely encode a confidential message in a wiretap channel. Our large-system characterization demonstrates that this secure coding scheme asymptotically achieves the secrecy capacity of the Gaussian wiretap channel.

The large-system performance of maximum-a-poste-rior estimation is studied considering a general distortion function when the observation vector is received through a linear system with additive white Gaussian noise. The analysis considers the system matrix to be chosen from the large class of rotationally invariant random matrices. We take a statistical mechanical approach by introducing a spin glass corresponding to the estimator, and employing the replica method for the large-system analysis. In contrast to earlier replica based studies, our analysis evaluates the general replica ansatz of the corresponding spin glass and determines the asymptotic distortion of the estimator for any structure of the replica correlation matrix. Consequently, the replica symmetric as well as the replica symmetry breaking ansatz with$b$steps of breaking is deduced from the given general replica ansatz. The generality of our distortion function lets us derive a more general form of the maximum-a-posterior decoupling principle. Based on the general replica ansatz, we show that for any structure of the replica correlation matrix, the vector-valued system decouples into a bank of equivalent decoupled scalar systems followed by maximum-a-posterior estimators. The structure of the decoupled system is further studied under both the replica symmetry and the replica symmetry breaking assumptions. For$b$steps of symmetry breaking, the decoupled system is found to be an additive system with anon-Gaussiannoise term given as the sum of an independent Gaussian random variable with$b$non-Gaussian impairment terms which depend on the input symbol. The general decoupling property of the maximum-a-posterior estimator leads to the idea of a replica simulator which represents the replica ansatz through the state evolution of a transition system described by its corresponding decoupled system. As an application of our study, we investigate large compressive sensing systems by considering the$\ell _{p}$norm minimization recovery schemes. Our numerical investigations show that the replica symmetric ansatz for$\ell _{0}$norm recovery fails to give an accurate approximation of the mean square error as the compression rate grows, and therefore, the replica symmetry breaking ansätze are needed in order to assess the performance precisely.

The large-system decoupling property of a MAP estimator is studied when it estimates the i.i.d. vector x from the observation y = Ax + z with A being chosen from a wide range of matrix ensembles, and the noise vector z being i.i.d. and Gaussian. Using the replica method, we show that the marginal joint distribution of any two corresponding input and output symbols converges to a deterministic distribution which describes the input-output distribution of a single user system followed by a MAP estimator. Under the bRSB assumption, the single user system is a scalar channel with additive noise where the noise term is given by the sum of an independent Gaussian random variable and b correlated interference terms. As the bRSB assumption reduces to RS, the interference terms vanish which results in the formerly studied RS decoupling principle.