We study in this paper lower bounds for the generalization error of models derived from multi-layer neural networks, in the regime where the size of the layers is commensurate with the number of samples in the training data. We derive explicit generalization lower bounds for general biased estimators, in the cases of two-layered networks. For linear activation function, the bound is asymptotically tight. In the nonlinear case, we provide a comparison of our bounds with an empirical study of the stochastic gradient descent algorithm. In addition, we derive bounds for unbiased estimators, which show that the latter have unacceptable performance for truly nonlinear networks. The analysis uses elements from the theory of large random matrices.

Nakamoto invented the longest chain protocol, and claimed its security by analyzing the private double-spend attack, a race between the adversary and the honest nodes to grow a longer chain. But is it the worst attack? We answer the question in the affirmative for three classes of longest chain protocols, designed for different consensus models: 1) Nakamoto's original Proof-of-Work protocol; 2) Ouroboros and SnowWhite Proof-of-Stake protocols; 3) Chia Proof-of-Space protocol. As a consequence, exact characterization of the maximum tolerable adversary power is obtained for each protocol as a function of the average block time normalized by the network delay. The security analysis of these protocols is performed in a unified manner by a novel method of reducing all attacks to a race between the adversary and the honest nodes.

We consider the following detection problem: given a realization of a symmetric matrix X of dimension n, distinguish between the hypothesis that all upper triangular variables are independent and identically distributed (i.i.d). Gaussians variables with mean 0 and variance 1 and the hypothesis, where X is the sum of such matrix and an independent rank-one perturbation. This setup applies to the situation, where under the alternative, there is a planted principal submatrix B of size L for which all upper triangular variables are i.i.d. Gaussians with mean 1 and variance 1, whereas all other upper triangular elements of X not in B are i.i.d. Gaussians variables with mean 0 and variance 1. We refer to this as the "Gaussian hidden clique problem." When L = (1 + ε)√n (ε > 0), it is possible to solve this detection problem with probability 1 - on(1) by computing the spectrum of X and considering the largest eigenvalue of X. We prove that this condition is tight in the following sense: when L <; (1 - ε)√n no algorithm that examines only the eigenvalues of X can detect the existence of a hidden Gaussian clique, with error probability vanishing as n → ∞. We prove this result as an immediate consequence of a more general result on rank-one perturbations of k-dimensional Gaussian tensors. In this context, we establish a lower bound on the critical signal-to-noise ratio below which a rank-one signal cannot be detected.

We consider the following detection problem: given a realization of asymmetric matrix $X$ of dimension $n$, distinguish between the hypothesisthat all upper triangular variables are i.i.d. Gaussians variableswith mean 0 and variance $1$ and the hypothesis that there is aplanted principal submatrix $B$ of dimension $L$ for which all upper triangularvariables are i.i.d. Gaussians with mean $1$ and variance $1$, whereasall other upper triangular elements of $X$ not in $B$ are i.i.d.Gaussians variables with mean 0 and variance $1$. We refer to this asthe `Gaussian hidden clique problem'. When $L=( 1 + \epsilon) \sqrt{n}$ ($\epsilon > 0$), it is possible to solve thisdetection problem with probability $1 - o_n(1)$ by computing thespectrum of $X$ and considering the largest eigenvalue of $X$.We prove that when$L < (1-\epsilon)\sqrt{n}$ no algorithm that examines only theeigenvalues of $X$can detect the existence of a hiddenGaussian clique, with error probability vanishing as $n \to \infty$.The result above is an immediate consequence of a more general result on rank-oneperturbations of $k$-dimensional Gaussian tensors.In this context we establish a lower bound on the criticalsignal-to-noise ratio below which a rank-one signal cannot be detected.

In this paper, we apply the theory of random Schrödinger operators to the analysis of multiusers communication channels similar to the Wyner model, which are characterized by short-range intercell interference. WithHthe channel transfer matrix,HHfis a narrow band matrix, a fact that does not permit the use of classical random matrices theory. On the other hand,HHfis in many aspects similar to a random Schrödinger operator. We relate the per-cell sum-rate capacity of the channel to the integrated density of states of a random Schrödinger operator; the latter is then related to the top Lyapunov exponent of a random sequence of matrices via a version of the Thouless formula. We also derive several bounds on the limiting per-cell sum-rate capacity, some based on the theory of random Schrödinger operators, and some derived from information theoretical considerations. Finally, we get explicit results in the high-signal-to-noise ratio (SNR) regime for some particular cases.

In this paper we study the spectrum of certain large random HermitianJacobimatrices. These matrices are known to describe certain communication setups. In particular, we are interested in an uplink cellular channel which models mobile users experiencing a soft-handoff situation under joint multicell decoding. Considering rather general fading statistics we provide a closed-form expression for the per-cell sum-rate of this channel in high signal-to-noise ratio (SNR), when an intra-cell time-division multiple-access (TDMA) protocol is employed. Since the matrices of interest aretridiagonal, their eigenvectors can be considered as sequences with second-order linear recurrence. Therefore, the problem is reduced to the study of the exponential growth of products of two-by-two matrices. For the case whereKusers are simultaneously active in each cell, we obtain a series of lower and upper bound on the high-SNR power offset of the per-cell sum-rate, which are considerably tighter than previously known bounds.

In this paper we study the spectrum of certain large random Hermitian Jacobi matrices. These matrices are known to describe certain communication setups. In particular we are interested in an uplink cellular channel which models mobile users experiencing a soft-handoff situation under joint multicell decoding. Considering rather general fading statistics we provide a closed form expression for the per-cell sum-rate of this channel in high-SNR, when an intra-cell TDMA protocol is employed. Since the matrices of interest are tridiagonal, their eigenvectors can be considered as sequences with second order linear recurrence. Therefore, the problem is reduced to the study of the exponential growth of products of two by two matrices. For the case where K users are simultaneously active in each cell, we obtain a series of lower and upper bounds on the high-SNR power offset of the per-cell sum-rate, which are considerably tighter than previously known bounds.

Curvature flows have been extensively considered from a deterministic point of view. They have been shown to be useful for a number of applications including crystal growth, flame propagation, and computer vision. In some previous work G. Ben-Arous et al. (2002), we have described a random particle system, evolving on the discretized unit circle, whose profile converges toward the Gauss-Minkowsky transformation of solutions of curve shortening flows initiated by convex curves. The present note shows that this theory may be implemented as a new way of evolving curves and as a possible alternative to level set methods.

We study the signal-to-interference (SIR) performance of linear multiuser receivers in random environments, where signals from the users arrive in "random directions." Such a random environment may arise in a DS-CDMA system with random signature sequences, or in a system with antenna diversity where the randomness is due to channel fading. Assuming that such random directions can be tracked by the receiver, the resulting SIR performance is a function of the directions and therefore also random. We study the asymptotic distribution of this random performance in the regime where both the number of users K and the number of degrees of freedom N in the system are large, but keeping their ratio fixed. Our results show that for both the decorrelator and the minimum mean-square error (MMSE) receiver, the variance of the SIR distribution decreases like 1/N, and the SIR distribution is asymptotically Gaussian. We compute closed-form expressions for the asymptotic means and variances for both receivers. Simulation results are presented to verify the accuracy of the asymptotic results for finite-sized systems.

Precise analytical asymptotic exponential rates of error, and bounds on those rates, for differential multiplephase-shift keying (DMPSK) systems that include post-detection integration are provided. Easily computed bounds on these rates are provided, both in the case of floor bit error probability (i.e., with no additive noise) and in the case of weak additive noise. The derivation uses the theory of large deviations and illustrates its applicability to the analysis of communications systems.>

The authors prove by means of a counterexample that it is not sufficient, for probably approximately correct (PAC) learning under a class of distributions, to have a uniform bound on the metric entropy of the class of concepts to be learned. This settles a conjecture of Benedek and Itai (1991).>

We consider the problem of classifying an unknown concept into one of two subclasses of concepts. Specifically, if C is a concept class and Co and Cl are two disjoint subsets of C, given an unknown c E Co U Cl we wish to decide whether c c Co or c E Cl based on a set of random examples. We consider both uniform and non-uniform probably correct classification for which the number of samples is or is not required to be independent of c, respectively. For both cases, we obtain necessary and sufficient conditions on Co and Cl that allow probably correct classification. The conditions obtained are in terms of separability and/or coverability conditions on the classes Co and Cl. Furthermore, in the non-uniform case we show that this is equivalent to classification in the limit. Several examples of the applicability of our results are also provided.

An extension of the standard probably approximately correct (PAC) learning model that allows the use of generalized samples is introduced. A generalized sample is viewed as a pair consisting of a functional on the concept class together with the value obtained by the functional operating on the unknown concept. It appears that this model can be applied to a number of problems in signal processing and geometric reconstruction to provide sample size bounds under a PAC criterion. A specific application of the generalized model to a problem of curve reconstruction is considered, and some connections with a result from stochastic geometry are discussed.>

In this paper, we introduce an extension of the standard PAC learning model which allows the use of generalized samples. We view a generalized sample as a pair consisting of a functional on the concept class together with the value obtained by the functional operating on the unknown concept. It appears that this model can be applied to a number of problems in signal processing and geometric reconstruction to provide sample size bounds under a PAC criterion. We consider a specific application of the model to a problem of curve reconstruction, and discuss some connections with a result from stochastic geometry.

The problem of deciding whether a sample of a random field was generated by a Gaussian distribution is considered. Based on extensions of large deviation estimates due to M.D. Donsker and S.R.S. Varadhan (1985), a test that is optimal in a generalized Neyman-Pearson sense is proposed. This test turns out to depend on properties of the entropy of Gaussian processes and does not depend on cumulant computations.>

The generalized likelihood ratio test (GLRT), which is commonly used in composite hypothesis testing problems, is investigated. Conditions for asymptotic optimality of the GLRT in the Neyman-Pearson sense are studied and discussed. First, a general necessary and sufficient condition is established, and then based on this, a sufficient condition, which is easier to verify, is derived. A counterexample where the GLRT is not optimal, is provided as well. A conjecture is stated concerning the optimality of the GLRT for the class of finite-state sources.>

A theorem characterizing fractional Brownian motion by the covariance structure of its wavelet transform is established. The authors examine whether there are alternate Gaussian processes whose wavelet transforms have a natural covariance structure. In addition, the authors examine if there are any Gaussian processes whose wavelet transform is stationary with respect to the affine group (i.e. the statistics of the wavelet transform do not depend on translations and dilations of the process).>

A prototype problem in hypotheses testing is discussed. The problem of deciding whether an i.i.d. sequence of random variables has originated from a known source P/sub 1/ or an unknown source P/sub 2/ is considered. The exponential rate of decrease in type II probability of error under a constraint on the minimal rate of decrease in type I probability of error is chosen for a criterion of optimality. Using large deviations estimates, a decision rule that is based on the relative entropy of the empirical measure with respect to P/sub 1/ is proposed. In the case of discrete random variables, this approach yields weaker results than the combinatorial approach used by Hoeffding (1965). However, it enables the analysis to be extended to the general case of R/sup n/-valued random variables. Finally, the results are extended to the case where P/sub 1/ is an unknown parameter-dependent distribution that is known to belong to a set of distributions (P/sup 0//sub 1/, theta in Theta ).>

New upper and lower bounds for the nonlinear filtering problem are presented. The lower bounds are especially useful in the region of small diffusion coefficients where previously known bounds are inefficient. The upper and lower bounds are shown to be tight. An example demonstrating the tightness of the bounds is presented.>

The problem of signal estimation from a measurement of its noise crossings is considered, following a nonlinear filtering approach that is different from the classical Davenport method. By focusing on a specific noise model and applying recent results on the excursions of a Brownian motion, an exact representation of the optimal filter is derived. Various extensions of the results are considered.>

The problem of estimating the number of transitions of finite-state continuous-time Markov processes observed by a noisy sensor is considered. A finite-dimensional exact filter is derived, and using the EM algorithm (an extension of the Baum-Welch algorithm for the discrete-time case), an application is made to the problem of estimating the unknown transition matrix of a finite-state continuous-time Markov process.>