Kernel ridge regression (KRR) has recently attracted renewed interest due to its potential for explaining the transient effects, such as double descent, that emerge during neural network training. In this work, we study how the alignment between the target function and the kernel affects the performance of the KRR. We focus on the truncated KRR (TKRR) which utilizes an additional parameter that controls the spectral truncation of the kernel matrix. We show that for polynomial alignment, there is an over-aligned regime, in which TKRR can achieve a faster rate than what is achievable by full KRR. The rate of TKRR can improve all the way to the parametric rate, while that of full KRR is capped at a sub-optimal value. This shows that target alignemnt can be better leveraged by utilizing spectral truncation in kernel methods. We also consider the bandlimited alignment setting and show that the regularization surface of TKRR can exhibit transient effects including multiple descent and non-monotonic behavior. Our results show that there is a strong and quantifable relation between the shape of the alignment spectrum and the generalization performance of kernel methods, both in terms of rates and in finite samples.

The general linear model (GLM) is a well established tool for analyzing functional magnetic resonance imaging (fMRI) data. Most fMRI analyses via GLM proceed in a massively univariate fashion where the same design matrix is used for analyzing data from each voxel. A major limitation of this approach is the locally varying nature of signals of interest as well as associated confounds. This local variability results in a potentially large bias and uncontrolled increase in variance for the contrast of interest. The main contributions of this paper are two fold: 1) we develop a statistical framework that enables estimation of an optimal design matrix while explicitly controlling the bias variance decomposition over a set of potential design matrices and 2) we develop and validate a numerical algorithm for computing optimal design matrices for general fMRI data sets. The implications of this framework include the ability to match optimally the magnitude of underlying signals to their true magnitudes while also matching the "null" signals to zero size thereby optimizing both the sensitivity and specificity of signal detection. By enabling the capture of multiple profiles of interest using a single contrast (as opposed to an F-test) in a way that optimizes for both bias and variance enables the passing of first level parameter estimates and their variances to the higher level for group analysis which is not possible using F-tests. We demonstrate the application of this approach to in vivo pharmacological fMRI data capturing the acute response to a drug infusion, to task-evoked, block design fMRI and to the estimation of a haemodynamic response function (HRF) in event-related fMRI. Although developed with motivation from fMRI, our framework is quite general and has potentially wide applicability to a variety of disciplines.

MOTIVATION: Two practical realities constrain the analysis of microarray data, mass spectra from proteomics, and biomedical infrared or magnetic resonance spectra. One is the 'curse of dimensionality': the number of features characterizing these data is in the thousands or tens of thousands. The other is the 'curse of dataset sparsity': the number of samples is limited. The consequences of these two curses are far-reaching when such data are used to classify the presence or absence of disease. RESULTS: Using very simple classifiers, we show for several publicly available microarray and proteomics datasets how these curses influence classification outcomes. In particular, even if the sample per feature ratio is increased to the recommended 5-10 by feature extraction/reduction methods, dataset sparsity can render any classification result statistically suspect. In addition, several 'optimal' feature sets are typically identifiable for sparse datasets, all producing perfect classification results, both for the training and independent validation sets. This non-uniqueness leads to interpretational difficulties and casts doubt on the biological relevance of any of these 'optimal' feature sets. We suggest an approach to assess the relative quality of apparently equally good classifiers.