In this paper, under the assumption that the dimension is much larger than the sample size, i.e.,p≍nα, α > 1, we consider the sample covariance matricesQ= Σ1/2XX*Σ1/2, whereX= (xij) is ap×nrandom matrix with centered i.i.d. entries whose variances are (pn)−1/2, and Σ is the deterministic population covariance matrix. This ultra-high dimensional setting is increasingly relevant in various applications in information theory, communications, and signal processing, such as massive MIMO, compressed sensing, and large-scale sensor networks. We establish two classes of central limit theorems (CLTs) for the linear spectral statistics (LSS) ofQ, the global CLTs on the macroscopic scales where all the eigenvalues will be used, and the local CLTs on the mesoscopic scales where only part of the eigenvalues will be utilized. We prove that the LSS converge to some Gaussian processes whose mean and covariance functions, depending on Σ, the ratiop/n, and the test functions, can be identified explicitly on both macroscopic and mesoscopic scales. We also show that even though the global CLTs depend on the fourth cumulant ofxij, the local CLTs do not. Leveraging these results, we propose two classes of statistics for testing the structures of Σ, the global statistics and the local statistics, and analyze their superior power under general local alternatives. To our best knowledge, the local LSS testing statistics, which do not rely on the fourth moment ofxij, are used for the first time in hypothesis testing while the literature mostly uses the global statistics and requires prior knowledge of the fourth moment. Numerical simulations also confirm the accuracy and powerfulness of our proposed statistics and illustrate better performance compared to the existing methods in the literature.

We systematically study the spectrum of kernel-based graph Laplacian (GL) constructed from high-dimensional and noisy random point cloud in the nonnull setup. The problem is motived by studying the model when the clean signal is sampled from a manifold that is embedded in a low-dimensional Euclidean subspace, and corrupted by high-dimensional noise. We quantify how the signal and noise interact in different regions of signal-to-noise ratio (SNR), and report the resulting peculiar spectral behavior of GL. In addition, we explore the impact of chosen kernel bandwidth on the spectrum of GL over different regions of SNR, which lead to an adaptive choice of kernel bandwidth that coincides with the common practice in real data. This result paves the way to a theoretical understanding of how practitioners apply GL when the dataset is noisy.

Detection of the number of signals corrupted by high-dimensional noise is a fundamental problem in signal processing and statistics. This paper focuses on a general setting where the high-dimensional noise has an unknown complicated heterogeneous variance structure. We propose a sequential test which utilizes the edge singular values (i.e., the largest few singular values) of the data matrix. It also naturally leads to a consistent sequential testing estimate of the number of signals. We describe the asymptotic distribution of the test statistic in terms of the Tracy-Widom distribution. The test is shown to be accurate and have full power against the alternative, both theoretically and numerically. The theoretical analysis relies on establishing the Tracy-Widom law for a large class of Gram type random matrices with non-zero means and completely arbitrary variance profiles, which can be of independent interest.

We apply local laws of random matrices and free probability theory to study the spectral properties of two kernel-based sensor fusion algorithms, nonparametric canonical correlation analysis (NCCA) and alternating diffusion (AD), for two simultaneously recorded high dimensional datasets under the null hypothesis. The matrix of interest is the product of the kernel matrices associated with the databsets, which may not be diagonalizable in general. We prove that in the regime where dimensions of both random vectors are comparable to the sample size, if NCCA and AD are conducted using a smooth kernel function, then the first few nontrivial eigenvalues will converge to real deterministic values provided the datasets are independent Gaussian random vectors. Toward the claimed result, we also provide a convergence rate of eigenvalues of a kernel affinity matrix.