High-dimensional time series appear in many scientific setups, demanding a nuanced approach to model and analyze the underlying dependence structure. Theoretical advancements so far often rely on stringent assumptions regarding the sparsity of the underlying signal. In non-sparse regimes, analyses have primarily focused on linear regression models with the design matrix having independent rows. In this paper, we expand the scope by investigating a high-dimensional time series model wherein the number of features grows proportionally to the number of sampling points, without assuming sparsity in the signal. Specifically, we consider the stochastic regression model and derive a single-letter formula for the normalized mutual information between observations and the signal, as well as for minimum mean-square errors. We also empirically study the vector approximate message passing VAMP algorithm and show that, despite the lack of theoretical guarantees, its performance for inference in our time series model is robust and often statistically optimal.

We rigorously analyse fully-trained neural networks of arbitrary depth in the Bayesian optimal setting in the so-called proportional scaling regime where the number of training samples and width of the input and all inner layers diverge proportionally. We prove an information-theoretic equivalence between the Bayesian deep neural network model trained from data generated by a teacher with matching architecture, and a simpler model of optimal inference in a generalized linear model. This equivalence enables us to compute the optimal generalization error for deep neural networks in this regime. We thus prove the "deep Gaussian equivalence principle" conjectured in Cui et al. (2023). Our result highlights that in order to escape this "trivialisation" of deep neural networks (in the sense of reduction to a linear model) happening in the strongly overparametrized proportional regime, models trained from much more data have to be considered.

High-dimensional time series appear in many scientific setups, demanding a nuanced approach to model and analyze the underlying dependence structure. However, theoretical advancements so far often rely on stringent assumptions regarding the sparsity of the underlying signals. In this contribution, we expand the scope by investigating a high-dimensional time series model wherein the number of features grows proportionally to the number of sampling points, without assuming sparsity in the signal. Specifically, we consider the stochastic regression model and derive a single-letter formula for the normalized mutual information between observations and the signal. We also empirically study the vector approximate message passing (VAMP) algorithm and show that, despite a lack of theoretical guarantees, its performance for inference in our time series model is robust and often statistically optimal.

It is shown that the distribution of the estimated canonical correlation coefficients between the past and the future of a high-dimensional multivariate white noise sequence converges almost surely towards a limit distribution whose density is given in closed form. A sketch of proof, based on free probability technics, is provided. Finally, it is briefly explained how this result can be used to produce consistent uncorrelatedness tests in the high-dimensional context.