Abstract Datasets comprised of large sets of both predictor and outcome variables are becoming more widely used in research. In addition to the well-known problems of model complexity and predictor variable selection, predictive modelling with such large data also presents a relatively novel and under-studied challenge of outcome variable selection. Certain outcome variables in the data may not be adequately predicted by the given sets of predictors. In this paper, we propose the method of Sparse Multivariate Principal Covariates Regression that addresses these issues altogether by expanding the Principal Covariates Regression model to incorporate sparsity penalties on both of predictor and outcome variables. Our method is one of the first methods that perform variable selection for both predictors and outcomes simultaneously. Moreover, by relying on summary variables that explain the variance in both predictor and outcome variables, the method offers a sparse and succinct model representation of the data. In a simulation study, the method performed better than methods with similar aims such as sparse Partial Least Squares at prediction of the outcome variables and recovery of the population parameters. Lastly, we administered the method on an empirical dataset to illustrate its application in practice.

BACKGROUND: Data analysis methods are usually subdivided in two distinct classes: There are methods for prediction and there are methods for exploration. In practice, however, there often is a need to learn from the data in both ways. For example, when predicting the antibody titers a few weeks after vaccination on the basis of genomewide mRNA transcription rates, also mechanistic insights about the effect of vaccinations on the immune system are sought. Principal covariates regression (PCovR) is a method that combines both purposes. Yet, it misses insightful representations of the data as these include all the variables. RESULTS: Here, we propose a sparse extension of principal covariates regression such that the resulting solutions are based on an automatically selected subset of the variables. Our method is shown to outperform competing methods like sparse principal components regression and sparse partial least squares in a simulation study. Furthermore good performance of the method is illustrated on publicly available data including antibody titers and genomewide transcription rates for subjects vaccinated against the flu: the selected genes by sparse PCovR are higly enriched for immune related terms and the method predicts the titers for an independent test sample well. In comparison, no significantly enriched terms were found for the genes selected by sparse partial least squares and out-of-sample prediction was worse. CONCLUSIONS: Sparse principal covariates regression is a promising and competitive tool for obtaining insights from high-dimensional data. AVAILABILITY: The source code implementing our proposed method is available from GitHub, together with all scripts used to extract, pre-process, analyze, and post-process the data: https://github.com/katrijnvandeun/SPCovR .

We start from a few examples of coupled behavioral sciences data, along with associated research questions and data-analytic methods. Linking up with these, we introduce a few concepts and distinctions, by means of which we specify the focus of this paper: 1) data that take the form of a collection of coupled matrices that are linked in either the experimental unit or the variable data mode; 2) associated with questions about the mechanisms underlying these data matrices; 3) which are to be addressed by data-analytic methods that rely on a submodel per data matrix, with a common parameterization of the shared data mode. Next, we outline the principles of two closely related families within this focus: the families of multiblock component- and factor-based models for data fusion (while considering both deterministic and stochastic model variants). Then, we review developments within these families to capture both similarities and differences between the different data matrices under study. We follow with a discussion on recent attempts to address quite a few challenges in data fusion based on multiblock component and factor models, including whether and how to differentially weigh the different data matrices under study, and problems such as dealing with large numbers of variables, outliers, and missing values. While the focus of this paper is on data and modeling contributions from the behavioral sciences, we point in a concluding section at their relevance for other domains and at the importance of related methods developed in those domains.