High-dimensional vector autoregressive (VAR) models have numerous applications in fields such as econometrics, biology, climatology, among others. While prior research has mainly focused on linear VAR models, these approaches can be restrictive in practice. To address this, we introduce a high-dimensional non-parametric sparse additive model, providing a more flexible framework. Our method employs basis expansions to construct high-dimensional nonlinear VAR models. We derive convergence rates and model selection consistency for least squared estimators, considering dependence measures of the processes, error moment conditions, sparsity, and basis expansions. Our theory significantly extends prior linear VAR models by incorporating both non-Gaussianity and non-linearity. As a key contribution, we derive sharp Bernstein-type inequalities for tail probabilities in both non-sub-Gaussian linear and nonlinear VAR processes, which match the classical Bernstein inequality for independent random variables. Additionally, we present numerical experiments that support our theoretical findings and demonstrate the advantages of the nonlinear VAR model for a gene expression time series dataset.

This paper develops asymptotic theory for quantile estimation via stochastic gradient descent (SGD) with a constant learning rate. The quantile loss function is neither smooth nor strongly convex. Beyond conventional perspectives and techniques, we view quantile SGD iteration as an irreducible, periodic, and positive recurrent Markov chain, which cyclically converges to its unique stationary distribution regardless of the arbitrarily fixed initialization. To derive the exact form of the stationary distribution, we analyze the structure of its characteristic function by exploiting the stationary equation.We also derive tight bounds for its moment generating function (MGF) and tail probabilities. Synthesizing the aforementioned approaches, we prove that the centered and standardized stationary distribution converges to a Gaussian distribution as the learning rate η → 0. This finding provides the first central limit theorem (CLT)-type theoretical guarantees for the quantile SGD estimator with constant learning rates. We further propose a recursive algorithm to construct confidence intervals of the estimators with statistical guarantees. Numerical studies demonstrate the effective finite-sample performance of the online estimator and inference procedure. The theoretical tools developed in this study are of independent interest for investigating general SGD algorithms formulated as Markov chains, particularly in non-strongly convex and non-smooth settings.

For time series with long-range temporal dependence, inference for covariance and precision matrices is non-trivial. We propose a Berry-Esseen type Gaussian approximation result that gives a finite-sample bound for the Kolmogorov distance between the infinity norms of the estimation error of sample covariance matrix and the corresponding Gaussian approximation. The method utilizes martingale andm-dependent approximation and relies on constructing triadic blocks. We also establish a bootstrapping result with block sampling method, which preserves validity despite strong temporal dependence. Our results on covariance allow ultra-high-dimensional settings where the dimension of time series can grow sub-exponentially with sample size. Similar results can be built for precision matrix under low-dimensional settings. No assumption is required on the structure of covariance and precision matrices.

Federated Learning has gained traction in privacy-sensitive collaborative environments, with local SGD emerging as a key optimization method in decentralized settings. While its convergence properties are well-studied, asymptotic statistical guarantees beyond convergence remain limited. In this paper, we present two generalized Gaussian approximation results for local SGD and explore their implications. First, we prove a Berry-Esseen theorem for the final local SGD iterates, enabling valid multiplier bootstrap procedures. Second, motivated by robustness considerations, we introduce two distinct time-uniform Gaussian approximations for the entire trajectory of local SGD. The time-uniform approximations support Gaussian bootstrap-based tests for detecting adversarial attacks. Extensive simulations are provided to support our theoretical results.

Stochastic gradient descent (SGD) with polynomially decaying step‐sizes has long underpinned theoretical analyses, yielding a broad spectrum of statistically attractive guarantees. Yet in practice, such schedules find rare use due to their prohibitively slow convergence, revealing a persistent gap between theory and empirical performance. In this paper, we introduce a unified framework that quantifies the uncertainty of online SGD under arbitrary learning‐rate choices. In particular, we provide the first comprehensive convergence characterizations for two widely used but theoretically under-examined schemes—cyclical learning rates and linear decay to zero. Our results not only explain the observed behavior of these schedules but also facilitate principled tools for statistical inference and algorithm design. All theoretical findings are corroborated by extensive simulations across diverse settings.

Stochastic Gradient Descent (SGD) and its Ruppert–Polyak averaged variant (ASGD) lie at the heart of modern large-scale learning, yet their theoretical properties in high-dimensional settings are rarely understood. In this paper, we provide rigorous statistical guarantees for constant learning-rate SGD and ASGD in high-dimensional regimes. Our key innovation is to transfer powerful tools from high-dimensional time series to online learning. Specifically, by viewing SGD as a nonlinear autoregressive process and adapting existing coupling techniques, we prove the geometric-moment contraction of high-dimensional SGD for constant learning rates, thereby establishing asymptotic stationarity of the iterates. Building on this, we derive the $q$-th moment convergence of SGD and ASGD for any $q\ge2$ in general $\ell^s$-norms, and, in particular, the $\ell^{\infty}$-norm that is frequently adopted in high-dimensional sparse or structured models. Furthermore, we provide sharp high-probability concentration analysis which entails the probabilistic bound of high-dimensional ASGD. Beyond closing a critical gap in SGD theory, our proposed framework offers a novel toolkit for analyzing a broad class of high-dimensional learning algorithms.

We establish a comprehensive finite-sample and asymptotic theory for stochastic gradient descent (SGD) with constant learning rates. First, we propose a novel linear approximation technique to provide a quenched central limit theorem (CLT) for SGD iterates with refined tail properties, showing that regardless of the chosen initialization, the fluctuations of the algorithm around its target point converge to a multivariate normal distribution. Our conditions are substantially milder than those required in the classical CLTs for SGD, yet offering a stronger convergence result. Furthermore, we derive the first Berry-Esseen bound -- the Gaussian approximation error -- for the constant learning-rate SGD, which is sharp compared to the decaying learning-rate schemes in the literature. Beyond the moment convergence, we also provide the Nagaev-type inequality for the SGD tail probabilities by adopting the autoregressive approximation techniques, which entails non-asymptotic large-deviation guarantees. These results are verified via numerical simulations, paving the way for theoretically grounded uncertainty quantification, especially with non-asymptotic validity.

This paper considers the recursive estimation of quantiles using the stochastic gradient descent (SGD) algorithm with Polyak-Ruppert averaging. The algorithm offers a computationally and memory efficient alternative to the usual empirical estimator. Our focus is on studying the non-asymptotic behavior by providing exponentially decreasing tail probability bounds under mild assumptions on the smoothness of the density functions. This novel non-asymptotic result is based on a bound of the moment generating function of the SGD estimate. We apply our result to the problem of best arm identification in a multi-armed stochastic bandit setting under quantile preferences.

In this paper, we study the concentration property of stochastic gradient descent (SGD) solutions. In existing concentration analyses, researchers impose restrictive requirements on the gradient noise, such as boundedness or sub-Gaussianity. We consider a much richer class of noise where only finitely-many moments are required, thus allowing heavy-tailed noises. In particular, we obtain Nagaev type high-probability upper bounds for the estimation errors of averaged stochastic gradient descent (ASGD) in a linear model. Specifically, we prove that, after $T$ steps of SGD, the ASGD estimate achieves an $O(\sqrt{\log(1/\delta)/T} + (\delta T^{q-1})^{-1/q})$ error rate with probability at least $1-\delta$, where $q>2$ controls the tail of the gradient noise. In comparison, one has the $O(\sqrt{\log(1/\delta)/T})$ error rate for sub-Gaussian noises. We also show that the Nagaev type upper bound is almost tight through an example, where the exact asymptotic form of the tail probability can be derived. Our concentration analysis indicates that, in the case of heavy-tailed noises, the polynomial dependence on the failure probability $\delta$ is generally unavoidable for the error rate of SGD.

This paper presents a solution to the real-time detection of MAC layer misbehaviors in IEEE 802.11 networks. Among the wide range of misbehaviors, we focus on the sender side selfish behavior that creates a channel- capturing effect by using favorable parameters, and the receiver side selfish behavior that does not respond with CTS and ACK upon receiving RTS and data packets, which clears the channel for itself and causes its sender to waste resources. These misbehaviors are subtle to detect, and yet can undermine the performance of the well-behaved nodes significantly. This paper shows a powerful real-time detection method that can catch these misbehaviors as soon as they have started. The detection method requires collecting delay, throughput, and packet interval data to generate time series and applying a sequential change point detection algorithm on the data streams as soon as new data points come in. All attacks are simulated in ns-3 and the simulation results verified the effectiveness of the detection method.

High-order cumulants and high-order spectra play an important role in the theory of stationary processes. For nonlinear processes, it is quite challenging to develop an asymptotic theory for their estimators. This paper presents a systematic asymptotic theory for estimators of high-order moments for a general class of stationary processes, using the framework of functional dependence measures. In particular, we prove the asymptotic normality of the estimators and establish a uniform convergence rate. We also provide a sufficient condition for the summability of high-order cumulants. Based on the latter, we prove consistency and asymptotic normality of the third-order spectra or bispectrum estimators under mild and easily verifiable conditions.

Due to the open nature of wireless communication medium, wireless networks are susceptible to jamming attacks. Jammers interfere with the legitimate nodes by sending strong jamming signals. Legitimate nodes can successfully transmit only between the gaps of the jamming signals. It is therefore very important to detect a jamming attack as soon as it happens in order to effectively take counter measurements. There are various types of jamming attacks, however, the {\it signature} of all jamming attacks is the performance degradation of legitimate nodes. Based on this observation, we develop a detection method using time series analysis approach. We model the network measurements taken over time as time series, and employ a sequential change point detection algorithm to detect the change of state in the time series, which is an indicator of change in the network state. Timely and accurate detection is the first step before further identification and localization of the source of interference. In this paper, we address the detection part and leave the localization of the jammer to future work. The jamming attacks are simulated in ns-3 simulator, and the detection result is satisfactory in terms of false alarm rate and detection delay.

This paper addresses detecting in-band wormholes in wireless ad hoc networks. The detection scheme requires collecting the end-to-end delay of packets at the receiver and then applying a sequential change point detection algorithm to detect abrupt changes in the delay time series. A new change point detection algorithm, named SW-CLT, is proposed. The algorithm is based on the Central Limit Theorem (CLT) and does not involve using a preset detecting threshold. The algorithm is compared with the non-parametric cumulative sum (NP-CUSUM) because the non-parametric version is believed to be more robust to highly dynamic data than the parametric version. SW-CLT has the ability to adjust its detection threshold with the variance of the data, and therefore is more robust than NP-CUSUM, which uses a preset threshold. Simulation results from ns3 verified the advantage of SW-CLT over NP-CUSUM in all simulated scenarios.

This paper considers the problem of sensor node localization, where a total of n anchor nodes are used to determine the locations of other nodes based on the received signal strengths. Challenges arise when anchor nodes are sparse and locations of them are not at grid positions. A range-based machine learning algorithm is developed to tackle the challenges. Instead of using samples to calibrate the parameters of a chosen signal model, we use machine learning to estimate the signal propagation function and its parameters at the same time. It overcomes the model dependency issue of existing range-based algorithms, and avoids the insufficient support issue of support vector machine methods. Simulation results show that the proposed algorithm has good adaptability to different signal characteristics, network deployment, and device variability. It significantly outperforms existing methods, especially when the anchor nodes are sparsely and irregularly deployed.

We consider the problem of identifying the source of failure in a network after receiving alarms or having observed symptoms. To locate the root cause accurately and timely in a large communication system is challenging because a single fault can often result in a large number of alarms, and multiple faults can occur concurrently. In this paper, we present a new fault localization method using a machine-learning approach. We propose to use logistic regression to study the correlation among network events based on end-to-end measurements. Then based on the regression model, we develop fault hypothesis that best explains the observed symptoms. Unlike previous work, the machine-learning algorithm requires neither the knowledge of dependencies among network events, nor the probabilities of faults, nor the conditional probabilities of fault propagation as input. The “low requirement” feature makes it suitable for large complex networks where accurate dependencies and prior probabilities are difficult to obtain. We then evaluate the performance of the learning algorithm with respect to the accuracy of fault hypothesis and the concentration property. Experimental results and theoretical analysis both show satisfactory performance.

When the grid topology is changed due to incidents and the state estimator is not updated with the topological change, it is considered a topology error. In this paper, we develop a new method for detecting topology errors in power grids. The proposed method considers the measurement data as a nonstationary Gaussian process, explores the dependence structure of the underlying process. It detects errors by testing the hypothesis of whether the mean vector of a nonstationary Gaussian process is zero and does not rely on the convergence of the standard weighted least-squares (WLS) state estimation algorithm. It is very effective in detecting topology errors, in which multiple conforming errors may occur and the traditional state estimation algorithms may fail to converge. Simulation results show that it can accurately identify the abnormal measurements caused by the topology error.

This paper considers online kernel estimation for both short- and long-range dependent time series data. Utilizing the predictive dependence measure of Wu, we carefully study the asymptotic properties of recursive kernel density and regression estimators for a general class of stationary processes. In particular, we prove that the proposed estimators have the asymptotic normality and the corresponding central limit theorems are provided. In addition, we establish the sharp laws of the iterated logarithms that precisely characterize the asymptotic almost sure behavior of the proposed estimators.

We propose a single-pass algorithm for estimating spectral densities of stationary processes. Our algorithm is computationally fast in the sense that, when a new observation arrives, it can provide a real-time update withinO(1) computation. The proposed algorithm is probabilistically fast in that, for stationary processes whose auto-covariances decay geometrically, the estimates from the algorithm converge at a rate which is optimal up to a multiplicative logarithmic factor. We also establish asymptotic normality for the recursive estimate. A simulation study is carried out and it confirms the superiority over the classical batched mean estimates.

We consider the problem of predicting aggregates or sums of future values of a process based on its past values. In contrast with the conventional prediction problem in which one predictsafuture value given past values of the process, in our setting the number of aggregates can go to infinity with respect to the number of available observations. Consistency and Bahadur representations of the prediction estimators are established. A simulation study is carried out to assess the performance of different prediction estimators.

An algorithm for generating sample paths of linear fractional stable motion (LFSM) is introduced. It is based on the approximation of LFSM by a linear process and exhibits low computational complexity. A detailed analysis of the error term involved in the approximation is provided, which in turn guides the user on selecting the size of the generated sequence.

We characterize the performance of difference coding for compressing sets and database relations through an analysis of the problem of estimating the number of bits needed for storing the spacings between values in sets of integers. We provide analytical expressions for estimating the effectiveness of difference coding when the elements of the sets or the attribute fields in database tuples are drawn from the uniform and Zipf distributions. We also examine the case where a uniformly distributed domain is combined with a Zipf distribution, and with an arbitrary distribution. We present limit theorems for most cases, and probabilistic convergence results in other cases. We also examine the effects of attribute domain reordering on the compression ratio. Our simulations show excellent agreement with theory.

We set up a general framework for evaluating the accuracy of the Markov model for the packet transmission over a wireless channel. Unlike most previous studies, a broad range of fading/shadowing models and coding/modulation schemes are included in our framework. We study the effect of both the packet length and mobile speed. Our analysis is based on the approximate normal distribution and the second order statistics in both the time and frequency domain. We make the generalized conclusion that the Markov model is most accurate when the mobile movement is either very fast (packets i.i.d.) or very slow (packets highly correlated). For the first time, we explicitly show that at intermediate speed when the correlation length in the fading is comparable with the packet length, the accuracy of the Markov model is highly dependent on the specific bit level model. This intermediate range changes with the packet length. A long packet will make this range narrow and occur at low speed.

We derive the exact statistical distribution of maximum a posteriori (MAP) estimators having edge-preserving nonGaussian priors. Such estimators have been widely advocated for image restoration and reconstruction problems. Previous investigations of these image recovery methods have been primarily empirical; the distribution we derive enables theoretical analysis. The signal model is linear with Gaussian measurement noise. We assume that the energy function of the prior distribution is chosen to ensure a unimodal posterior distribution (for which convexity of the energy function is sufficient), and that the energy function satisfies a uniform Lipschitz regularity condition. The regularity conditions are sufficiently general to encompass popular priors such as the generalized Gaussian Markov random field prior and the Huber prior, even though those priors are not everywhere twice continuously differentiable.