Variational Gaussian processes (GPs) approximate exact GP inference by using a small set of inducing points to form a sparse approximation of the true posterior, with the fidelity of the model increasing with additional inducing points. Although the approximation error in principle can be reduced through the use of more inducing points, this leads to scaling optimization challenges and computational complexity. To achieve scalability, inducing point methods typically introduce conditional independencies and then approximations to the training and test conditional distributions. In this paper, we consider an alternative approach to modifying the training and test conditionals, in which we make them more flexible. In particular, we investigate decoupling the parametric form of the predictive mean and covariance in the conditionals, and learn independent parameters for predictive mean and covariance. We derive new evidence lower bounds (ELBO) under these more flexible conditionals, and provide two concrete examples of applying the decoupled conditionals. Empirically, we find this additional flexibility leads to improved model performance on a variety of regression tasks and Bayesian optimization (BO) applications.

Effective collaborative discourse requires both cognitive and social engagement of students. To investigate complex socio-cognitive dynamics in collaborative discourse, this paper proposes to model collaborative discourse as a socio-semantic network (SSN) and then use network motifs – defined as recurring, significant subgraphs – to characterize the network and hence the discourse. To demonstrate the utility of our SSN motifs framework, we applied it to a sample dataset. While more work needs to be done, the SSN motifs framework shows promise as a novel, theoretically informed approach to discourse analysis.

Vegetation index (VI) derived from remotely sensed images is a proxy of terrestrial vegetation information and widely used in land monitoring and global change studies. Recently, the prediction of vegetation properties has been an interest in related communities. With the accumulation of satellite records over the past few decades, the spatial–temporal prediction of VI becomes feasible. In this letter, we developed deep recurrent neural networks (RNNs) with long short-term memory (LSTM) and gated recurrent units (GRUs) to predict the short-term VI based on historical observations. The pixel-based fully connected networks GRU and LSTM (FCGRU and FCLSTM) and patch-based convolutional networks (ConvGRU and ConvLSTM) are established and compared with the traditional multilayer perceptron (MLP) model. Moderate Resolution Imaging Spectroradiometer (MODIS) and Sentinel-2 normalized difference VI (NDVI) data sets were used in the experiments. The prediction performance is evaluated globally in different regions, different vegetation types, and different growing seasons. Results demonstrate that the RNN models can predict VI with high accuracy (average root mean square error (RMSE) around 0.03), which is superior to the MLP model. In general, the pixel-based RNN models performed better than the patch-based models especially in regions with a larger proportion of outliers. And the prediction accuracy is stable over different vegetation types and growing seasons.

Spatiotemporally continuous long-term leaf area index (LAI) products are urgently needed to monitor environmental changes. The current filter- or curve-fitting-based time series reconstructive algorithms fail to reconstruct the LAI time series with many continuous missing values or missing values in key phenological periods, which are common issues in high-spatial-resolution LAI time series. In this article, a meteorological data-driven backpropagation neural network (MBPNN) was proposed to reconstruct discontinuous LAI profiles with a two-step process using vegetation phenological information. As the basis of the strong dependence of vegetation growth on meteorological conditions, a reasonable growth trajectory of reconstructed LAI can be guaranteed by the algorithm even though if many observed values are missing. Validations for reconstructed LAI were conducted both spatially and temporally based on reference maps and field-measured long-term observations. The results showed that the LAI predicted by the MBPNN had a similar accuracy (RMSE = 0.4076) as the Landsat LAI inversions (RMSE = 0.4083) and a similar reconstructed trajectory as the field-measured LAI series even though over 100 days of continuous data were missing (RMSE = 0.1620). A comparison with the Harmonic ANalysis of Time Series (HANTS) algorithm showed that the accuracy of MBPNN was more stable regardless of the size/position of the missing data, and the proposed method performed much better when the data were continuously missing for 50 days or more.

Gaussian processes with derivative information are useful in many settings where derivative information is available, including numerous Bayesian optimization and regression tasks that arise in the natural sciences. Incorporating derivative observations, however, comes with a dominating $O(N^3D^3)$ computational cost when training on $N$ points in $D$ input dimensions. This is intractable for even moderately sized problems. While recent work has addressed this intractability in the low-$D$ setting, the high-$N$, high-$D$ setting is still unexplored and of great value, particularly as machine learning problems increasingly become high dimensional. In this paper, we introduce methods to achieve fully scalable Gaussian process regression with derivatives using variational inference. Analogous to the use of inducing values to sparsify the labels of a training set, we introduce the concept of inducing directional derivatives to sparsify the partial derivative information of the training set. This enables us to construct a variational posterior that incorporates derivative information but whose size depends neither on the full dataset size $N$ nor the full dimensionality $D$. We demonstrate the full scalability of our approach on a variety of tasks, ranging from a high dimensional Stellarator fusion regression task to training graph convolutional neural networks on PubMed using Bayesian optimization. Surprisingly, we additionally find that our approach can improve regression performance even in settings where only label data is available.

Multitask learning has proven to be useful in the field of machine learning when additional knowledge is available to help a prediction task. We adapt this paradigm to develop autotuning frameworks, where the objective is to find the optimal performance parameters of an application code that is treated as a black-box function. Furthermore, we combine multitask learning with multi-objective tuning and incorporation of coarse performance models to enhance the tuning capability. The proposed framework is parallelized and applicable to any application, particularly exascale applications with a small number of function evaluations. Compared with other state-of-the-art single-task learning frameworks, the proposed framework attains up to 2.8X better code performance for at least 80% of all tasks using up to 2048 cores.

High-quality Leaf Area Index (LAI) time-series is important for many ecological applications. Unfortunately, troubles of observations missing and low spatial-temporal resolution greatly restrict their further applications. Due to the increasing of algorithm uncertainty, the current time-series optimized (TSO) algorithms perform poorly when LAI observations are lost continuously or unavailable on the key phenology nodes. It is an effective way of improving performance of TSO by introducing prior knowledge which replenishes time-series detail information. Meteorological data is completely competent since its great potential of describing vegetation growing rules. In this paper, focusing on data missing trouble, we develop a new LAI time-series reconstruction algorithm, called MNNR (Meteorology and Neural Network based Reconstruction), by introducing external meteorological data and other prior information into a neural network model. The results demonstrate that the proposed MNNR algorithm is well capable of spatial-temporal LAI reconstruction and performs excellently when observations are lost continuously.