The rapid growth of the Internet of Things (IoT) has enabled the development of intelligent transportation systems (ITS). However, missing data remains a common issue that hinders the effectiveness of ITS. Although many imputation methods have been proposed to handle missing values, they generally aim to improve the accuracy of final imputation results and often neglect the interpretability of the imputation process. This may cause limited trustworthiness and practical applicability in real-world metro management, where understanding the underlying metro data patterns is essential. To address this gap, this paper conducts an in-depth analysis and mining of the unique spatiotemporal structures inherent in metro flow data, aiming at both accurate and interpretable imputation. Specifically, an orthogonal-autoregression matrix factorization (OAMF) algorithm is proposed to discover two low-dimensional, complete spatial and temporal patterns from high-dimensional, incomplete data. By minimizing reconstruction error associated with these two patterns, OAMF ensures the accuracy of final imputation results. Furthermore, through simultaneous distinctive spatial representations modeling and predictive temporal modeling, OAMF encourages the two patterns with clear physical meaning that captures orthogonal spatial distributions and evolving metro flow trends over time, enabling the interpretability of the imputation process. Additionally, the above explicit temporal modeling further extends OAMF’s ability to predict futures. The proposed OAMF is validated using two real-world datasets. Experimental results demonstrate that OAMF not only extracts interpretable spatiotemporal patterns but also achieves superior accuracy in both imputation and prediction across various missing rates compared to existing methods.

Modeling nonlinear dynamical systems is a challenging task in fields such as speech processing, music generation, and video prediction. This paper introduces a hierarchical framework for Deep State Space Models (DSSMs), categorizing them by their conditional independence properties and Markov assumptions and positioning existing models within this framework, including the Stochastic Recurrent Neural Network (SRNN), Variational Recurrent Neural Network (VRNN), and Recurrent State Space Model (RSSM). We discuss different options for the inference networks and demonstrate how integrating normalizing flows can enhance model flexibility by capturing complex distributions. Our work not only clarifies the relationships among existing models but also paves the way for the development of new, more effective approaches for modeling nonlinear dynamics. In particular, we propose the Autoregressive State Space Model (ArSSM) and evaluate its effectiveness in speech and polyphonic music modeling tasks.

This paper presents Deep Dynamic Probabilistic Canonical Correlation Analysis (D2PCCA), a model that integrates deep learning with probabilistic modeling to analyze nonlinear dynamical systems. Building on the probabilistic extensions of Canonical Correlation Analysis (CCA), D2PCCA captures nonlinear latent dynamics and supports enhancements such as KL annealing for improved convergence and normalizing flows for a more flexible posterior approximation. D2PCCA naturally extends to multiple observed variables, making it a versatile tool for encoding prior knowledge about sequential datasets and providing a probabilistic understanding of the system’s dynamics. Experimental validation on real financial datasets demonstrates the effectiveness of D2PCCA and its extensions in capturing latent dynamics.

Fine-tuning large-scale pre-trained models provides an effective solution to alleviate the label scarcity problem in cardiovascular diseases (CVDs) detection using electrocardiogram (ECG). However, as the pre-trained models scale up, the computational costs for fine-tuning and inference become unaffordable on low-level devices deployed for clinical applications. Additionally, maintaining the model performance under low budgets in computational resources remains a significant challenge. However, a comprehensive study that can address them in a joint framework is still lacking. Here, we propose a holistic method (H-Tuning) for low-cost and efficient fine-tuning of pre-trained models on downstream datasets. Then, the inference costs of the models fine-tuned by H-Tuning are further reduced significantly using a knowledge distillation technique. Experiments on four ECG datasets demonstrate that H-Tuning reduces the GPU memory consumption during fine-tuning by 6.34 times while achieving comparable CVDs detection performance to standard fine-tuning. With the knowledge distillation technique, the model inference latency and the memory consumption are reduced by 4.52 times and 19.83 times. As such, the proposed joint framework allows for the utilization of pre-trained models with high computation efficiency and robust performance, exploring a path toward low-cost and efficient CVDs detection. Code is available at https://github.com/KAZABANA/H-Tuning

In closed-loop Industrial Internet of Things (IIoT), not all deviations lead to the process out of control due to the compensation of controllers. Normal operating condition changes, dynamic anomalies with unchanged operating conditions, and complete loss of control can be finely distinguished by monitoring static or dynamic components of the process. However, existing methods often suffer from overlapping decomposition of dynamic and static components with limited interpretability, which causes information leak and reduces monitoring performance. To address these issues, this paper presents an orthogonal projection to dynamic latent structure (OP-DLS) method, enabling fine-grained process monitoring that distinguishes different types of deviations. Specifically, by jointly minimizing the dynamic unpredictable variance and the static reconstruction error, OP-DLS maps process data onto non-overlapped dynamic and static subspaces through an orthogonal projection to prevent information leak. In addition, the temporal dependence and representative properties of dynamic components are explicitly modeled for better generalization and interpretability. Building upon this, targeted monitoring indices are designed to monitor the dynamic, static, and global aspects of the process for fine-grained distinction of different types of deviations. Compared with existing methods, the superiority of our proposed method is validated through a numerical example and an industrial-scale three-phase flow facility process.

The scarcity of labeled data poses a significant challenge in the field of data-driven rotating machinery fault diagnosis. In practical applications, although there is a large amount of available data, only a very small portion is labeled. Unfortunately, few studies have investigated leveraging such extremely limited labeled data to improve diagnostic performance. To address this issue, a clustering-guided framework is proposed to embed the extremely limited label information into deep clustering for diagnosis performance improvement. Specifically, a clustering-guided feature learning module is proposed to unify feature learning and clustering, enabling the learned features to have optimal clustering characteristics. Meanwhile, the prior knowledge given by a few labeled data is integrated into clustering centroid initialization and cluster assignment to further improve the clustering accuracy and robustness. By introducing a virtual adversarial training regularization, the generalization of the method is enhanced. Experiment results on four datasets show that the proposed method outperforms the existing methods, which demonstrates the effectiveness of the proposed clustering-guided framework in dealing with the extreme data scarcity problem in fault diagnosis for rotating machinery.

Monitoring nonlinear dynamic processes is challenging due to the inherent complexity and uncertainty in industrial systems. Fault detection and fault identification are critical components of process monitoring, yet existing methods often struggle to capture predictable dynamics with compact model parameters. This could compromise the fault detection performance and further impede the fault identification accuracy. This article proposes a unified modeling framework to strategically address these challenges through nonlinearity handling, most predictable dynamics extraction, and parsimonious system parameterization. Building on this enhanced capability, the proposed model integrates fault detection and identification with mutual information, thereby improving the identification accuracy. Validations on the revamped Tennessee Eastman Process and a real-world multiphase flow facility demonstrate superior fault detection metrics and fault identification capabilities compared to alternative approaches.

Electroencephalography (EEG) is an objective tool for emotion recognition and shows promising performance. However, the label scarcity problem is a main challenge in this field, which limits the wide application of EEG-based emotion recognition. In this article, we propose a novel semisupervised transfer learning framework (EEGMatch) to leverage both labeled and unlabeled EEG data. First, an EEG-Mixup-based data augmentation method is developed to generate more valid samples for model learning. Second, a semisupervised two-step pairwise learning method is proposed to bridge prototypewise and instancewise pairwise learning, where the prototypewise pairwise learning measures the global relationship between EEG data and the prototypical representation of each emotion class and the instancewise pairwise learning captures the local intrinsic relationship among EEG data. Third, a semisupervised multidomain adaptation is introduced to align the data representation among multiple domains (labeled source domain, unlabeled source domain, and target domain), where the distribution mismatch is alleviated. Extensive experiments are conducted on three benchmark databases (SEED, SEED-IV, and SEED-V) under a cross-subject leave-one-subject-out cross-validation evaluation protocol. The results show the proposed EEGMatch performs better than the state-of-the-art methods under different incomplete label conditions (with 5.89% improvement on SEED, 0.93% improvement on SEED-IV, and 0.28% improvement on SEED-V), which demonstrates the effectiveness of the proposed EEGMatch in dealing with the label scarcity problem in emotion recognition using EEG signals. The source code is available at https://github.com/KAZABANA/EEGMatch.

In traditional multivariate data analysis, dimension reduction and regression have been treated as distinct endeavors. Established techniques such as principal component regression (PCR) and partial least squares (PLS) regression traditionally compute latent components as intermediary steps—although with different underlying criteria—before proceeding with the regression analysis. In this paper, we introduce an innovative regression methodology named PLS-integrated Lasso (PLS-Lasso) that integrates the concept of dimension reduction directly into the regression process. We present two distinct formulations for PLS-Lasso, denoted as PLS-Lasso-v1 and PLS-Lasso-v2, along with clear and effective algorithms that ensure convergence to global optima. PLS-Lasso-v1 and PLS-Lasso-v2 are compared with Lasso on the task of financial index tracking and show promising results.

Electrocardiography (ECG) is a non-invasive tool for predicting cardiovascular diseases (CVDs). Current ECG-based diagnosis systems show promising performance owing to the rapid development of deep learning techniques. However, the label scarcity problem, the co-occurrence of multiple CVDs and the poor performance on unseen datasets greatly hinder the widespread application of deep learning-based models. Addressing them in a unified framework remains a significant challenge. To this end, we propose a multi-label semi-supervised model (ECGMatch) to recognize multiple CVDs simultaneously with limited supervision. In the ECGMatch, an ECGAugment module is developed for weak and strong ECG data augmentation, which generates diverse samples for model training. Subsequently, a hyperparameter-efficient framework with neighbor agreement modeling and knowledge distillation is designed for pseudo-label generation and refinement, which mitigates the label scarcity problem. Finally, a label correlation alignment module is proposed to capture the co-occurrence information of different CVDs within labeled samples and propagate this information to unlabeled samples. Extensive experiments on four datasets and three protocols demonstrate the effectiveness and stability of the proposed model, especially on unseen datasets. As such, this model can pave the way for diagnostic systems that achieve robust performance on multi-label CVDs prediction with limited supervision.

Affective brain-computer interface based on electroencephalography (EEG) is an important branch in the field of affective computing. However, the individual differences in EEG emotional data and the noisy labeling problem in the subjective feedback seriously limit the effectiveness and generalizability of existing models. To tackle these two critical issues, we propose a novel transfer learning framework with Prototypical Representation based Pairwise Learning (PR-PL). The discriminative and generalized EEG features are learned for emotion revealing across individuals and the emotion recognition task is formulated as pairwise learning for improving the model tolerance to the noisy labels. More specifically, a prototypical learning is developed to encode the inherent emotion-related semantic structure of EEG data and align the individuals' EEG features to a shared common feature space under consideration of the feature separability of both source and target domains. Based on the aligned feature representations, pairwise learning with an adaptive pseudo labeling method is introduced to encode the proximity relationships among samples and alleviate the label noises effect on modeling. Extensive results on two benchmark databases (SEED and SEED-IV) under four different cross-validation evaluation protocols validate the model reliability and stability across subjects and sessions. Compared to the literature, the average enhancement of emotion recognition across four different evaluation protocols is 2.04% (SEED) and 2.58% (SEED-IV). The source code is available athttps://github.com/KAZABANA/PR-PL.

Intelligent transfer domain adaptation fault diagnosis approaches have received more and more attention. An important prerequisite for the effectiveness of the existing transfer diagnosis methods is to have complete target machine data of all failure types for domain adaptation. However, in practical applications, it is often difficult and time-consuming to obtain all kinds of target fault data. In addition, the diagnosis model cannot be updated to adapt to the actual operating conditions. To overcome these limitations, we propose a time-varying online transfer learning (TVOTL) model for intelligent fault diagnosis of rolling bearings, which can not only construct the transfer diagnosis model with incomplete unlabeled target data offline but also update dynamically during online application. In the offline stage, the initial feature transformation matrix is first obtained using limited inter-domain data and the raw data are mapped to the corresponding feature space to get new representations. Then, in the online stage, data distribution discrepancies between domains are further reduced with online collected target instances and the transfer diagnosis model is updated. In the case study, the proposed TVOTL model is tested on ten transfer diagnosis tasks to illustrate the outstanding performance compared with other online fault diagnosis models.

Dynamic-inner canonical correlation analysis (DiCCA) extracts dynamic latent variables from high-dimensional time series data with a descending order of predictability in terms of R2. The reduced dimensional latent variables with rank-ordered predictability capture the dynamic features in the data, leading to easy interpretation and visualization. In this article, numerically efficient algorithms for DiCCA are developed to extract dynamic latent components from high-dimensional time series data. The numerically improved DiCCA algorithms avoid repeatedly inverting a covariance matrix inside the iteration loop of the numerical DiCCA algorithms. A further improvement using singular value decomposition converts the generalized eigenvector problem into a standard eigenvector problem for the DiCCA solution. Another improvement in model efficiency in this article is the dynamic model compaction of the extracted latent scores using autoregressive integrated moving average (ARIMA) models. Integrating factors, if existed in the latent variable scores, are made explicit in the ARIMA models. Numerical tests on two industrial datasets are provided to illustrate the improvements.