High-dimensional time series appear in many scientific setups, demanding a nuanced approach to model and analyze the underlying dependence structure. Theoretical advancements so far often rely on stringent assumptions regarding the sparsity of the underlying signal. In non-sparse regimes, analyses have primarily focused on linear regression models with the design matrix having independent rows. In this paper, we expand the scope by investigating a high-dimensional time series model wherein the number of features grows proportionally to the number of sampling points, without assuming sparsity in the signal. Specifically, we consider the stochastic regression model and derive a single-letter formula for the normalized mutual information between observations and the signal, as well as for minimum mean-square errors. We also empirically study the vector approximate message passing VAMP algorithm and show that, despite the lack of theoretical guarantees, its performance for inference in our time series model is robust and often statistically optimal.

High-dimensional time series appear in many scientific setups, demanding a nuanced approach to model and analyze the underlying dependence structure. However, theoretical advancements so far often rely on stringent assumptions regarding the sparsity of the underlying signals. In this contribution, we expand the scope by investigating a high-dimensional time series model wherein the number of features grows proportionally to the number of sampling points, without assuming sparsity in the signal. Specifically, we consider the stochastic regression model and derive a single-letter formula for the normalized mutual information between observations and the signal. We also empirically study the vector approximate message passing (VAMP) algorithm and show that, despite a lack of theoretical guarantees, its performance for inference in our time series model is robust and often statistically optimal.

Over the past decade, characterizing the precise asymptotic risk of regularized estimators in high-dimensional regression has emerged as a prominent research area. This literature focuses on the proportional asymptotics regime, where the number of features and samples diverge proportionally. Much of this work assumes i.i.d. Gaussian entries in the design. Concurrently, researchers have explored the universality of these findings, discovering that results based on the i.i.d. Gaussian assumption extend to other settings, including i.i.d. sub-Gaussian designs. However, universality results examining dependent covariates have predominanatly focused on correlation-based dependence or structured forms of dependence allowed by right-rotationally-invariant designs. In this paper, we challenge this limitation by investigating dependence structures beyond these established classes. We identify a class of designs characterized by a block dependence structure where results based on i.i.d. Gaussian designs persist. Formally, we establish that the optimal values of regularized empirical risk and the risk associated with convex regularized estimators, such as the Lasso and the ridge, converge to the same limit under block-dependent designs as for i.i.d. Gaussian entry designs. Our dependence structure differs significantly from correlation-based dependence and enables, for the first time, asymptotically exact risk characterization in prevalent high-dimensional nonlinear regression problems.