Spatio-temporal kriging is an essential research problem in sensor networks due to the sparsity of deployed sensors. While recent studies consider spatial and temporal correlations, they often overlook the sparse spatial distribution of locations and the incomplete features across locations. To tackle these problems, we propose an Anchor-based Incremental and Stratified Graph Learning Framework for Inductive Spatio-Temporal Kriging (AnchorGK). AnchorGK introduces anchor locations to enable effective data stratification for accurate kriging. Anchor locations are constructed based on feature availability, and strata are subsequently established based on the an- chor locations. This stratification serves two purposes: 1) it ensures that the spatial correlations between unknown areas (no observations) and surrounding known locations are accurately represented and dynamically updated within the graph learning framework, and 2) it facilitates the use of all available features across different strata through a novel incremental representation method. Building on the data stratification, we propose a dual-view graph learning layer that integrates information from relevant features and locations and learns distinct representations for different strata. Finally, kriging is performed based on the obtained strata representations. Experimental results on multiple benchmark datasets demonstrate that AnchorGK consistently outperforms existing state-of-the-art methods. Our codes, datasets, and related materials are given in: https://github.com/xren451/Spatial-interpolation

Time series prediction presents a significant challenge across various domains, such as transportation systems, environmental science, and multiple industrial sectors. Real-world time series data commonly exhibit periodic patterns and irregular sampling rates. Recent advancements in long sequence time series forecasting have made significant progress in adopting deep neural networks, particularly the Transformers, renowned for their robust representational capabilities. However, current Transformer-based models consider time steps as discrete tokens, thereby failing to account for periodicity and temporal intervals when selecting relevant time steps in the past. To address this limitation, we propose an end-to-end framework called Periormer for forecasting irregularly sampled time series. Periormer comprises three key components: (1) a novel input embedding layer that encodes the periodicity and time interval information, analogous to positional encoding in Transformers; (2) a feature-wise periodic attention mechanism that selects essential data points considering the periods and amplitudes of the periodic signals; and (3) a cross-feature periodic attention mechanism that identifies essential features relevant to the prediction. Experiments on four real-world datasets and one synthetic dataset demonstrate that Periormer reduces the mean squared error by 14.9% compared to state-of-the-art models.

A growing number of tasks require adaptive machine learning systems capable of learning continuously from incoming data and adapting to changes in their environment. In order to enable the widespread adoption of machine learning for streaming data, it is crucial that practitioners and researchers have the tools to efficiently build and evaluate adaptive learning systems. In this paper we demonstrate FALL, a Framework for Adaptive Life-long Learning, which we have developed to enable the full adaptive learning pipeline to be built using modular, reusable components, enabling users to easily and efficiently develop, implement, and evaluate state-of-the-art adaptive learning systems. Source code, documentation, and examples may be found at https://benhalstead.dev/FALL/.

Missing data imputation for location-based sensor data has attracted much attention in recent years. The state-of-the-art imputation methods based on graph neural networks have a priori assumption that the spatial correlations between sensor locations are static. However, real-world data sets often exhibit dynamic spatial correlations. This paper proposes a novel approach to capturing the dynamics of spatial correlations between geographical locations as a composition of the constant, long-term trends and periodic patterns. To this end, we design a new method called Dynamic Adjacency Matrix Representation (DAMR) that extracts various dynamic patterns of spatial correlations and represents them as adjacency matrices. The adjacency matrices are then aggregated and fed into a well-designed graph representation learning layer for predicting the missing values. Through extensive experiments on six real-world data sets, we demonstrate that DAMR reduces the MAE by up to 19.4% compared with the state-of-the-art methods for the missing value imputation task

Learning from streaming data is challenging as the distribution of incoming data may change over time, a phenomenon known as concept drift. The predictive patterns, or experience learned under one distribution may become irrelevant as conditions change under concept drift, but may become relevant once again when conditions reoccur. Adaptive learning methods adapt a classifier to concept drift by identifying which distribution, or concept , is currently present in order to determine which experience is relevant. Identifying a concept requires some representation to be stored for comparison, with the quality of the representation being key to accurate identification. Existing concept representations are based on meta-features, efficient univariate summaries of a concept. However, no single meta-feature can fully represent a concept, leading to severe accuracy loss when existing representations cannot describe concept drift. To avoid these failure cases, we propose the first general framework for combining a diverse range of meta-features into a single representation. We solve two main challenges, first presenting a method of efficiently computing, storing, and querying an arbitrary set of meta-features as a single representation, showing that a combination of meta-features may successfully avoid failure cases seen with existing methods. Second, we present the first method for dynamically learning which meta-features distinguish concepts in any given dataset, significantly improving performance. Our proposed approach enables state-of-the-art feature selection methods, such as mutual information, to be applied to concept representation meta-features for the first time. We investigate tradeoffs between memory budget and classification performance, observing accuracy increases of up to 16% by dynamically weighting the contribution of each meta-feature.

The distribution of streaming data often changes over time as conditions change, a phenomenon known as concept drift. Only a subset of previous experience, collected in similar conditions, is relevant to learning an accurate classifier for current data. Learning from irrelevant experience describing a different concept can degrade performance. A system learning from streaming data must identify which recent experience is irrelevant when conditions change and which past experience is relevant when concepts reoccur, e.g., when weather events or financial patterns repeat. Existing streaming approaches either do not consider experience to change in relevance over time and thus cannot handle concept drift, or only consider the recency of experience and thus cannot handle recurring concepts, or only sparsely evaluate relevance and thus fail when concept drift is missed. To enable learning in changing conditions, we propose SELeCT, a probabilistic method for continuously evaluating the relevance of past experience. SELeCT maintains a distinct internal state for each concept, representing relevant experience with a unique classifier. We propose a Bayesian algorithm for estimating state relevance, combining the likelihood of drawing recent observations from a given state with a transition pattern prior based on the system’s current state. The current state is continuously maintained using a Hoeffding bound based algorithm, which unlike existing methods, guarantees that every observation is classified using the state estimated as the most relevant, while also maintaining temporal stability. We find SELeCT is able to choose experience relevant to ground truth concepts with recall and precision above 0.9, significantly outperforming existing methods and close to a theoretical optimum, leading to significantly higher accuracy and enabling new opportunities for learning in complex changing conditions.

Data collected over time often exhibit changes in distribution, or concept drift, caused by changes in hidden context relevant to the classification task, e.g. weather conditions. Adaptive learning methods are able to retain performance in changing conditions by explicitly detecting concept drift and changing the classifier used to make predictions. However, in realworld conditions, existing methods often select classifiers which poorly represent current data due to adaptation errors, where change in context is misidentified. We propose the AiRStream system, which uses a novel repair algorithm to identify and correct adaptation errors. We identify errors by periodically testing the performance of inactive classifiers. If an error is identified, a backtracking procedure repairs training done under the misidentified context. AiRStream achieves higher accuracy compared to baseline methods and selects classifiers which better match changes in context. A case study on a real-world air quality inference task shows that AiRStream is able to build a robust model of environmental conditions, allowing the adaptions made to concept drift to be analysed and related to changes in weather.

Streaming sources of data are becoming more common as the ability to collect data in real-time grows. A major concern in dealing with data streams is concept drift, a change in the distribution of data over time, for example, due to changes in environmental conditions. Representing concepts (stationary periods featuring similar behaviour) is a key idea in adapting to concept drift. By testing the similarity of a concept representation to a window of observations, we can detect concept drift to a new or previously seen recurring concept. Concept representations are constructed using meta-information features, values describing aspects of concept behaviour. We find that previously proposed concept representations rely on small numbers of meta-information features. These representations often cannot distinguish concepts, leaving systems vulnerable to concept drift. We propose FiCSUM, a general framework to represent both supervised and unsupervised behaviours of a concept in a fingerprint, a vector of many distinct meta-information features able to uniquely identify more concepts. Our dynamic weighting strategy learns which meta-information features describe concept drift in a given dataset, allowing a diverse set of meta-information features to be used at once. FiCSUM outperforms state-of-the-art methods over a range of 11 real world and synthetic datasets in both accuracy and modeling underlying concept drift.

Machine Learning typically assumes that training and test sets are independently drawn from the same distribution, but this assumption is often violated in practice which creates a bias. Many attempts to identify and mitigate this bias have been proposed, but they usually rely on ground-truth information. But what if the researcher is not even aware of the bias? In contrast to prior work, this paper introduces a new method, Imitate, to identify and mitigate Selection Bias in the case that we may not know if (and where) a bias is present, and hence no ground-truth information is available. Imitate investigates the dataset's probability density, then adds generated points in order to smooth out the density and have it resemble a Gaussian, the most common density occurring in real-world applications. If the artificial points focus on certain areas and are not widespread, this could indicate a Selection Bias where these areas are underrepresented in the sample. We demonstrate the effectiveness of the proposed method in both, synthetic and real-world datasets. We also point out limitations and future research directions.