Accurate time series forecasting is crucial for optimizing resource allocation, industrial production, and urban management, particularly with the growth of cyber-physical and IoT systems. However, limited training sample availability in fields like physics and biology poses significant challenges. Existing models struggle to capture long-term dependencies and to model diverse meta-knowledge explicitly in few-shot scenarios. To address these issues, we propose MetaGP, a meta-learning-based Gaussian process latent variable model that uses a Gaussian process kernel function to capture long-term dependencies and to maintain strong correlations in time series. We also introduce Kernel Association Search (KAS) as a novel meta-learning component to explicitly model meta-knowledge, thereby enhancing both interpretability and prediction accuracy. We study MetaGP on simulated and real-world few-shot datasets, showing that it is capable of state-of-the-art prediction accuracy. We also find that MetaGP can capture long-term dependencies and can model meta-knowledge, thereby providing valuable insights into complex time series patterns.

Trajectory similarity computation serves as a fundamental functionality of various spatial information applications. Although existing deep learning similarity computation methods offer better efficiency and accuracy than non-learning solutions, they are still immature in trajectory embedding and suffer from poor generality and heavy preprocessing for training. Targeting these limitations, we propose a novel framework named KGTS based on knowledge graph grid embedding, prompt trajectory embedding, and unsupervised contrastive learning for improved trajectory similarity computation. Specifically, we first embed map grids with a GRot embedding method to vigorously grasp the neighbouring relations of grids. Then, a prompt trajectory embedding network incorporates the resulting grid embedding and extracts trajectory structure and point order information. It is trained by unsupervised contrastive learning, which not only alleviates the heavy preprocessing burden but also provides exceptional generality with creatively designed strategies for positive sample generation. The prompt trajectory embedding adopts a customized prompt paradigm to mitigate the gap between the grid embedding and the trajectory embedding. Extensive experiments on two real-world trajectory datasets demonstrate the superior performance of KGTS over state-of-the-art methods.

Electroencephalogram (EEG)-based brain–computer interfaces (BCIs) allow users to use brain signals to control external instruments, and movement intention detecting BCIs can aid in the rehabilitation of patients who have lost motor function. Existing studies in this area mostly rely on cue-based data collection that facilitates sample labeling but introduces noise from cue stimuli; moreover, it requires extensive user training, and cannot reflect real usage scenarios. In contrast, self-paced BCIs can overcome the limitations of the cue-based approach by supporting users to perform movements at their own initiative and pace, but they fall short in labeling. Therefore, in this study, we proposed an automated labeling approach that can cross-reference electromyography (EMG) signals for EEG labeling with zero human effort. Furthermore, considering that only a few studies have focused on evaluating BCI systems for online use and most of them do not report details of the online systems, we developed and present in detail a pseudo-online evaluation suite to facilitate online BCI research. We collected self-paced movement EEG data from 10 participants performing opening and closing hand movements for training and evaluation. The results show that the automated labeling method can contend well with noisy data compared with the baseline labeling method. We also explored popular machine learning models for online self-paced movement detection. The results demonstrate the capability of our online pipeline, and that a well-performing offline model does not necessarily translate to a well-performing online model owing to the specific settings of an online BCI system. Our proposed automated labeling method, online evaluation suite, and dataset take a concrete step towards real-world self-paced BCI systems.

The pervasiveness of GPS-enabled devices and wireless communication technologies flourish the market of Spatial Crowdsourcing (SC), which consists of location-based tasks and requires workers to be at specific locations physically to complete them. In this work, we study the problem of worker-churn-based task assignment in SC, where tasks are assigned by considering workers’ churn. In particular, we aim to maximize the total rewards of task assignments based on the worker churn prediction. To solve the problem, we propose a two-phase framework, which consists of a worker churn prediction and a task assignment phase. In the first phase, we use an LSTM-based model to extract latent feelings of workers based on historical data and then estimate idle time intervals of workers. In the assignment phase, we design an efficient greedy algorithm and a Kuhn-Munkras-based algorithm that can achieve the optimal task assignment. To improve the accuracy of the idle time interval estimation for workers, we adopt a context-dependent LSTM model, which involves interactions between inputs and their context. We further optimize the original task assignment framework by proposing a travel distance optimization strategy to reduce the overall travel distance. Extensive experiments offer insight into the effectiveness and efficiency of the proposed solutions.

Ubiquitous mobile sensors on human activity recognition pose the threat of leaking personal information that is implicitly contained within the time-series sensor signals and can be extracted by attackers. Existing protective methods only support specific sensitive attributes and require massive relevant sensitive ground truth for training, which is unfavourable to users. To fill this gap, we propose a novel data transformation framework for prohibiting the leakage of sensitive information from sensor data. The proposed framework transforms raw sensor data into a new format, where the sensitive information is hidden and the desired information (e.g., human activities) is retained. Training can be conducted without using any personal information as ground truth. Meanwhile, multiple attributes of sensitive information (e.g., age, gender) can be collectively hidden through a one-time transformation. The experimental results on two multimodal sensor-based human activity datasets manifest the feasibility of the presented framework in hiding users’ sensitive information (inference MAE increases$\sim$2 times and inference accuracy degrades$\sim$50%) without degrading the usability of the data for activity recognition (only$\sim$2% accuracy degradation).

Brain-computer interface (BCI) is a system empowering humans to communicate with or control the outside world with exclusively brain intentions. Electroencephalography (EEG)-based BCI is one of the promising solutions due to its convenient and portable instruments. Despite the extensive research of EEG in recent years, it is still challenging to interpret EEG signals effectively due to its nature of noise and difficulties in capturing the inconspicuous relations between EEG signals and specific brain activities. Most existing works either only consider EEG as chain-like sequences while neglecting complex dependencies between adjacent signals or requiring complex preprocessing. In this paper, we introduce two deep learning-based frameworks with novel spatio-temporal preserving representations of raw EEG streams to precisely identify human intentions. The two frameworks consist of both convolutional and recurrent neural networks effectively exploring the preserved spatial and temporal information in either a cascade or a parallel manner. Extensive experiments on a large scale movement intention EEG dataset (108 subjects, 3 145 160 EEG records) have demonstrated that the proposed frameworks achieve high accuracy of 98.3% and outperform a set of state-of-the-art and baseline models. The developed models are further evaluated with a real-world brain typing BCI and achieve a recognition accuracy of 93% over five instruction intentions suggesting good generalization over different kinds of intentions and BCI systems.

Motor imagery classification from EEG signals is essential for motor rehabilitation with a Brain-Computer Interface (BCI). Most current works on this issue require a subject-specific adaptation step before applied to a new user. Thus the research of directly extending a pre-trained model to new users is particularly desired and indispensable. As brain dynamics fluctuate considerably across different subjects, it is challenging to design practical hand-crafted features based on prior knowledge. Regarding this gap, this paper proposes a Graph-based Convolutional Recurrent Attention Model (G-CRAM) to explore EEG features across different subjects for motor imagery classification. A graph structure is first developed to represent the positioning information of EEG nodes. Then a convolutional recurrent attention model learns EEG features from both spatial and temporal dimensions and emphasizes on the most distinguishable temporal periods. We evaluate the proposed approach on two benchmark EEG datasets of motor imagery classification on the subject-independent testing. The results show that the G-CRAM achieves superior performance to state-of-the-art methods regarding recognition accuracy and ROC-AUC. Furthermore, model interpretation studies reveal the learning process of different neural network components and demonstrate that the proposed model can extract detailed features efficiently.

Recent years have witnessed the success of deep learning methods in human activity recognition (HAR). The longstanding shortage of labeled activity data inherently calls for a plethora of semisupervised learning methods, and one of the most challenging and common issues with semisupervised learning is the imbalanced distribution of labeled data over classes. Although the problem has long existed in broad real-world HAR applications, it is rarely explored in the literature. In this paper, we propose a semisupervised deep model for imbalanced activity recognition from multimodal wearable sensory data. We aim to address not only the challenges of multimodal sensor data (e.g., interperson variability and interclass similarity) but also the limited labeled data and class-imbalance issues simultaneously. In particular, we propose a pattern-balanced semisupervised framework to extract and preserve diverse latent patterns of activities. Furthermore, we exploit the independence of multi-modalities of sensory data and attentively identify salient regions that are indicative of human activities from inputs by our recurrent convolutional attention networks. Our experimental results demonstrate that the proposed model achieves a competitive performance compared to a multitude of state-of-the-art methods, both semisupervised and supervised ones, with 10% labeled training data. The results also show the robustness of our method over imbalanced, small training data sets.

Semi-supervised learning is crucial for alleviating labelling burdens in people-centric sensing. However, humangenerated data inherently suffer from distribution shift in semi-supervised learning due to the diverse biological conditions and behavior patterns of humans. To address this problem, we propose a generic distributionally robust model for semi-supervised learning on distributionally shifted data. Considering both the discrepancy and the consistency between the labeled data and the unlabeled data, we learn the latent features that reduce person-specific discrepancy and preserve task-specific consistency. We evaluate our model in a variety of people-centric recognition tasks on real-world datasets, including intention recognition, activity recognition, muscular movement recognition and gesture recognition. The experiment results demonstrate that the proposed model outperforms the state-of-the-art methods.

As brain dynamics fluctuate considerably across different subjects, it is challenging to design effective handcrafted features based on prior knowledge. Regarding this gap, this paper proposes a Graph-based Convolutional Recurrent Attention Model (G-CRAM) to explore EEG features across different subjects for movement intention recognition. A graph structure is first developed to embed the positioning information of EEG nodes, and then a convolutional recurrent attention model learns EEG features from both spatial and temporal dimensions and adaptively emphasizes on the most distinguishable temporal periods. The proposed approach is validated on two public movement intention EEG datasets. The results show that the GCRAM achieves superior performance to state-of-the-art methods regarding recognition accuracy and ROC-AUC. Furthermore, model interpreting studies reveal the learning process of different neural network components and demonstrate that the proposed model can extract detailed features efficiently.

Sharing ubiquitous mobile sensor data, especially physiological data, raises potential risks of leaking physical and demographic information that can be inferred from the time series sensor data. Existing sensitive information protection mechanisms that depend on data transformation are effective only on a particular sensitive attribute, together with usually requiring the labels of sensitive information for training. Considering this gap, we propose a novel user sensitive information protection framework without using a sensitive training dataset or being validated on protecting only one specific sensitive information. The presented approach transforms raw sensor data into a new format that has a "style" (sensitive information) of random noise and a "content" (desired information) of the raw sensor data, thus is free of user sensitive information for training and able to collectively protect all sensitive information at once. Our implementation and experiments on two real-world multisensor human activity datasets demonstrate that the proposed data transformation technique can achieve the protection for all sensitive information at once without requiring the knowledge of users' personal attributes for training, and simultaneously preserve the usability of the new transformed data with regard to inferring human activities with insignificant performance loss.

Multi-modality is an important feature of sensor based activity recognition. In this work, we consider two inherent characteristics of human activities, the spatially-temporally varying salience of features and the relations between activities and corresponding body part motions. Based on these, we propose a multi-agent spatial-temporal attention model. The spatial-temporal attention mechanism helps intelligently select informative modalities and their active periods. And the multiple agents in the proposed model represent activities with collective motions across body parts by independently selecting modalities associated with single motions. With a joint recognition goal, the agents share gained information and coordinate their selection policies to learn the optimal recognition model. The experimental results on four real-world datasets demonstrate that the proposed model outperforms the state-of-the-art methods.

The electroencephalogram (EEG) signal is a medium to realize a brain-computer interface (BCI) system due to its zero clinical risk and portable acquisition devices. Current EEG-based BCI research usually requires a subject-specific adaptation step before a BCI can be employed by a new user. In contrast, the subject-independent scenario, where a well trained model can be directly applied to new users without precalibration, is particularly desired. Considering this critical gap, the focus in this letter is developing an effective EEG signal analysis adaptively applied to subject-independent settings. We present a convolutional recurrent attention model (CRAM) that utilizes a convolutional neural network to encode the high-level representation of EEG signals and a recurrent attention mechanism to explore the temporal dynamics of the EEG signals as well as to focus on the most discriminative temporal periods. Extensive experiments on a benchmark multiclass EEG dataset containing four movement intentions indicate that the proposed model is capable of exploiting the underlying invariant EEG patterns across different subjects and generalizing the patterns to new subjects with better performance than a series of state-of-the-art and baseline approaches by at least eight percentage points. The implementation code is made publicly available.11https://github.com/dalinzhang/CRAM.

Brain-Computer Interface (BCI) enables human to communicate with and intuitively control an external device through brain signals. Movement intention recognition paves the path for developing BCI applications. The current state-of-the-art in EEG based BCI usually involves subject-specific adaptation before ready to use. However, the subject-independent scenario, in which a well-trained model is directly applied to new subjects without any pre-calibration, is particularly desired yet rarely explored. In order to fill the gap, we present a Convolutional Attention Model (CAM) for EEG-based human movement intention recognition in the subject-independent scenario. The convolutional network is designed to capture the spatio-temporal features of EEG signals, while the integrated attention mechanism is utilized to focus on the most discriminative information of EEG signals during the period of movement imagination while omitting other less relative parts. Experiments conducted on a real-world EEG dataset containing 55 subjects show that our model is capable of mining the underlying invariant EEG patterns across different subjects and generalizing to unseen subjects. Our model achieves better performance than a series of state-of-the-art and baseline approaches.

Multimodal features play a key role in wearable sensor based human activity recognition (HAR). Selecting the most salient features adaptively is a promising way to maximize the effectiveness of multimodal sensor data. In this regard, we propose a “collect fully and select wisely” principle as well as an interpretable parallel recurrent model with convolutional attentions to improve the recognition performance. We first collect modality features and the relations between each pair of features to generate activity frames, and then introduce an attention mechanism to select the most prominent regions from activity frames precisely. The selected frames not only maximize the utilization of valid features but also reduce the number of features to be computed effectively. We further analyze the accuracy and interpretability of the proposed model based on extensive experiments. The results show that our model achieves competitive performance on two benchmarked datasets and works well in real life scenarios.