Industrial processes often generate complex, interdependent time-series data from multiple sensors across multiple stages, forming complex dependencies among variables and process stages. Effective monitoring and timely anomaly detection of these time series through multivariate time series anomaly detection (MTAD) is crucial for preventing failures and ensuring the reliability of automated systems. Graph neural networks (GNNs) have advanced MTAD by leveraging data-driven graphs to model complex dependencies among variables, effectively capturing relational structures within multivariate time series to enhance anomaly detection performance. However, existing GNN-based approaches often overlook critical process knowledge, and even when this knowledge is considered, seamlessly incorporating it into existing models remains inherently challenging, leading to suboptimal performance. To address this limitation, we propose a knowledge-assisted multi-graph framework for modeling sensor dependencies in multi-stage industrial processes for MTAD, which explicitly incorporates process knowledge into graph learning to enhance dependency modeling and improve anomaly detection performance. Our method constructs three complementary graphs: one purely data-driven and two refined by integrating structural constraints derived from process knowledge. To effectively leverage these graphs for anomaly detection, we employ a multi-graph attention network, enabling a more accurate and robust representation of complex dependencies. Comprehensive experiments on two real-world, multi-stage industrial datasets demonstrate that incorporating process knowledge substantially enhances anomaly detection performance. Note to Practitioners-This work tackles anomaly detection in multi-stage industrial processes, where many sensors exhibit dependencies both within and across stages. Methods that learn sensor relations purely from data often overlook critical dependencies or infer spurious ones. To address this limitation, the proposed approach incorporates two forms of commonly available process knowledge-sensor group knowledge (sensors associated with each sub-process) and process flow knowledge (adjacent sub-process relationships)-derived from process diagrams. The method builds three complementary graphs (uninformed, sensor group knowledge-informed, and process flow knowledge-informed) and uses a multi-graph attention network combined with a temporal convolutional network for forecasting-based anomaly detection. Experiments on water treatment and water distribution systems show substantial F1 improvements over data-driven baselines, ablation studies confirming the contributions of both knowledge sources. For deployment, practitioners only need to specify sensor groupings and stage adjacencies from existing diagrams; no manual relationship modeling or specialized domain expertise is required. The model is trained on normal-operation data and detects anomalies via prediction errors. The approach is most effective when sub-process boundaries and flows are well documented and should be updated if the plant configuration changes. Beyond water systems, this method can be applied to chemical processing, power generation, and automated manufacturing facilities with similar multi-stage structures.

Uncertainty quantification (UQ) is crucial for deploying machine learning models in high-stakes applications, where overconfident predictions can lead to serious consequences. An effective UQ method must balance computational efficiency with the ability to generalize across diverse scenarios. Evidential deep learning (EDL) achieves efficiency by modeling uncertainty through the prediction of a Dirichlet distribution over class probabilities. However, the restrictive assumption of Dirichlet-distributed class probabilities limits EDL's robustness, particularly in complex or unforeseen situations. To address this, we propose *flexible evidential deep learning* ($\mathcal{F}$-EDL), which extends EDL by predicting a flexible Dirichlet distribution—a generalization of the Dirichlet distribution—over class probabilities. This approach provides a more expressive and adaptive representation of uncertainty, significantly enhancing UQ generalization and reliability under challenging scenarios. We theoretically establish several advantages of $\mathcal{F}$-EDL and empirically demonstrate its state-of-the-art UQ performance across diverse evaluation settings, including classical, long-tailed, and noisy in-distribution scenarios.

Pseudo-label-based semi-supervised learning (SSL) algorithms trained on a class-imbalanced set face two cascading challenges: 1) Classifiers tend to be biased towards majority classes, and 2) Biased pseudo-labels are used for training. It is difficult to appropriately re-balance the classifiers in SSL because the class distribution of an unlabeled set is often unknown and could be mismatched with that of a labeled set. We propose a novel class-imbalanced SSL algorithm called class-distribution-mismatch-aware debiasing (CDMAD). For each iteration of training, CDMAD first assesses the classifier's biased degree towards each class by calculating the logits on an image without any patterns (e.g., solid color image), which can be considered irrelevant to the training set. CDMAD then refines biased pseudo-labels of the base SSL algorithm by ensuring the classifier's neutrality. CDMAD uses these refined pseudo-labels during the training of the base SSL algorithm to improve the quality of the representations. In the test phase, CDMAD similarly refines biased class predictions on test samples. CDMAD can be seen as an extension of post-hoc logit adjustment to address a challenge of incorporating the unknown class distribution of the unlabeled set for re-balancing the biased classifier under class distribution mismatch. CDMAD ensures Fisher consistency for the balanced error. Extensive experiments verify the effectiveness of CDMAD.

Existing deep learning methods for clustering high-dimensional data perform feature selection and clustering separately, which can result in the exclusion of some important features for clustering. In this paper, we propose a method that performs deep clustering and feature selection simultaneously by inserting a concrete selector layer between the input layer and the first encoder layer of a modified autoencoder. The concrete selector layer performs feature selection, while the modified autoencoder performs clustering in the latent space by incorporating K-means loss and inter-cluster distances. The proposed method, called the K-concrete autoencoder, selects features important for clustering and uses only the selected features to learn K-means-friendly latent representations of the data. Moreover, we propose an extension of the K-concrete autoencoder to provide relative importance of each selected feature. We demonstrate the effectiveness of the proposed method using simulated and real datasets.

Evidential deep learning (EDL) has shown remarkable success in uncertainty estimation. However, there is still room for improvement, particularly in out-of-distribution (OOD) detection and classification tasks. The limited OOD detection performance of EDL arises from its inability to reflect the distance between the testing example and training data when quantifying uncertainty, while its limited classification performance stems from its parameterization of the concentration parameters. To address these limitations, we propose a novel method called Density Aware Evidential Deep Learning (DAEDL). DAEDL integrates the feature space density of the testing example with the output of EDL during the prediction stage, while using a novel parameterization that resolves the issues in the conventional parameterization. We prove that DAEDL enjoys a number of favorable theoretical properties. DAEDL demonstrates state-of-the-art performance across diverse downstream tasks related to uncertainty estimation and classification.

Existing deep learning methods for clustering high-dimensional data perform feature selection and clustering separately, which can result in the exclusion of some important features for clustering. In this paper, we propose a method that performs deep clustering and feature selection simultaneously by inserting a concrete selector layer between the input layer and the first encoder layer of a modified autoencoder. The concrete selector layer performs feature selection, while the modified autoencoder performs clustering in the latent space by incorporating K-means loss and inter-cluster distances. The proposed method, called the K-concrete autoencoder, selects features important for clustering and uses only the selected features to learn K-means-friendly latent representations of the data. Moreover, we propose an extension of the K-concrete autoencoder to provide relative importance of each selected feature. We demonstrate the effectiveness of the proposed method using simulated and real datasets.

Recent studies have shown that the generalization ability of deep neural networks (DNNs) is closely related to the Fisher information matrix (FIM) calculated during the early training phase. Several methods have been proposed to regularize the FIM for increased generalization of DNNs. However, they cannot be used directly for Bayesian neural networks (BNNs) because the variable parameters of BNNs make it difficult to calculate the FIM. To address this problem, we achieve regularization of the FIM of BNNs by specifying a new suitable prior distribution called the inverse-reference (IR) prior. To regularize the FIM, the IR prior is derived as the inverse of the reference prior that imposes minimal prior knowledge on the parameters and maximizes the trace of the FIM. We demonstrate that the IR prior can enhance the generalization ability of BNNs for large-scale data over previously used priors while providing adequate uncertainty quantifications using various benchmark image datasets and BNN structures.

A Gaussian process is a standard surrogate model for an unknown objective function in Bayesian optimization. In this paper, we propose a new surrogate model, called the objective bound conditional Gaussian process (OBCGP), to condition a Gaussian process on a bound on the optimal function value. The bound is obtained and updated as the best observed value during the sequential optimization procedure. Unlike the standard Gaussian process, the OBCGP explicitly incorporates the existence of a point that improves the best known bound. We treat the location of such a point as a model parameter and estimate it jointly with other parameters by maximizing the likelihood using variational inference. Within the standard Bayesian optimization framework, the OBCGP can be combined with various acquisition functions to select the next query point. In particular, we derive cumulative regret bounds for the OBCGP combined with the upper confidence bound acquisition algorithm. Furthermore, the OBCGP can inherently incorporate a new type of prior knowledge, i.e., the bounds on the optimum, if it is available. The incorporation of this type of prior knowledge into a surrogate model has not been studied previously. We demonstrate the effectiveness of the OBCGP through its application to Bayesian optimization tasks, such as the sequential design of experiments and hyperparameter optimization in neural networks.

Several out-of-distribution (OOD) detection scores have been recently proposed for deep generative models because the direct use of the likelihood threshold for OOD detection has been shown to be problematic. In this paper, we propose a new OOD score based on a Bayesian hypothesis test called the locally most powerful Bayesian test (LMPBT). The LMPBT is locally most powerful in that the alternative hypothesis (the representative parameter for the OOD sample) is specified to maximize the probability that the Bayes factor exceeds the evidence threshold in favor of the alternative hypothesis provided that the parameter specified under the alternative hypothesis is in the neighborhood of the parameter specified under the null hypothesis. That is, under this neighborhood parameter condition, the test with the proposed alternative hypothesis maximizes the probability of correct detection of OOD samples. We also propose numerical strategies for more efficient and reliable computation of the LMPBT for practical application to deep generative models. Evaluations conducted of the OOD detection performance of the LMPBT on various benchmark datasets demonstrate its superior performance over existing OOD detection methods.

Existing semi-supervised learning (SSL) algorithms typically assume class-balanced datasets, although the class distributions of many real world datasets are imbalanced. In general, classifiers trained on a class-imbalanced dataset are biased toward the majority classes. This issue becomes more problematic for SSL algorithms because they utilize the biased prediction of unlabeled data for training. However, traditional class-imbalanced learning techniques, which are designed for labeled data, cannot be readily combined with SSL algorithms. We propose a scalable class-imbalanced SSL algorithm that can effectively use unlabeled data, while mitigating class imbalance by introducing an auxiliary balanced classifier (ABC) of a single layer, which is attached to a representation layer of an existing SSL algorithm. The ABC is trained with a class-balanced loss of a minibatch, while using high-quality representations learned from all data points in the minibatch using the backbone SSL algorithm to avoid overfitting and information loss. Moreover, we use consistency regularization, a recent SSL technique for utilizing unlabeled data in a modified way, to train the ABC to be balanced among the classes by selecting unlabeled data with the same probability for each class. The proposed algorithm achieves state-of-the-art performance in various class-imbalanced SSL experiments using four benchmark datasets.

We propose a few-shot learning method for detecting out-of-distribution (OOD) samples from classes that are unseen during training while classifying samples from seen classes using only a few labeled examples. For detecting unseen classes while generalizing to new samples of known classes, we synthesize fake samples, i.e., OOD samples, but that resemble in-distribution samples, and use them along with real samples. Our approach is based on an extension of model-agnostic meta learning (MAML) and is denoted as OOD-MAML, which not only learns a model initialization but also the initial fake samples across tasks. The learned initial fake samples can be used to quickly adapt to new tasks to form task-specific fake samples with only one or a few gradient update steps using MAML. For testing, OOD-MAML converts a K-shot N-way classification task into N sub-tasks of K-shot OOD detection with respect to each class. The joint analysis of N sub-tasks facilitates simultaneous classification and OOD detection and, furthermore, offers an advantage, in that it does not require re-training when the number of classes for a test task differs from that for training tasks; it is sufficient to simply assume as many sub-tasks as the number of classes for the test task. We also demonstrate the effective performance of OOD-MAML over benchmark datasets.

In complex industrial processes, process fault detection and classification constitute an important task for reducing production costs and improving product quality. Most existing methods for fault classification assume that sufficient labeled data are available for training. However, label acquisition is costly and laborious in practice, whereas abundant unlabeled data are often available. To make effective use of a large amount of unlabeled data for fault classification, we propose in this article a new approach using semi-supervised deep generative models, allowing the complex relationship between high-dimensional process data and the process status to be modeled. In particular, to consider the temporal correlation and intervariable correlation in multivariate time series process data collected from multiple sensors, we propose two semi-supervised deep generative models incorporating convolutional neural networks. The proposed models are assessed on data from the Tennessee Eastman benchmark process. The results demonstrate the superior performances of the proposed models compared with competing methods.

We propose a new architecture of the capsule network called the ladder capsule network, which has an alternative building block to the dynamic routing algorithm in the capsule network (Sabour et al., 2017). Motivated by the need for using only important capsules during training for robust performance, we first introduce a new layer called the pruning layer, which removes irrelevant capsules. Based on the selected capsules, we construct higher-level capsule outputs. Subsequently, to capture the part-whole spatial relationships, we introduce another new layer called the ladder layer, the outputs of which are regressed lower-level capsule outputs from higher-level capsules. Unlike the capsule network adopting the routing-by-agreement, the ladder capsule network uses backpropagation from a loss function to reconstruct the lower-level capsule outputs from higher-level capsules; thus, the ladder layer implements the reverse directional inference of the agreement/disagreement mechanism of the capsule network. The experiments on MNIST demonstrate that the ladder capsule network learns an equivariant representation and improves the capability to extrapolate or generalize to pose variations.

When a person seeks another person's attention, it is of prime importance to assess how interruptible the other person is. Since smartphones are ubiquitously used as communication media these days, interruptibility prediction on smartphones has started to attract great interest from both academia and industry. Previous studies, in general, attempted to model interruptibility using the behaviors at the current moment and in the immediate past (e.g., 5 minutes before). However, a person's interruptibility at a certain moment is indeed affected by his/her preceding behaviors for several reasons. Motivated by this long-term effect, in this paper we propose a novel methodology of extracting features based on past behaviors from smartphone sensor data. The primary difference from previous studies is that we systematically consider a longer history of up to a day in addition to the current point and the immediate past. To represent behaviors in a day accurately and compactly, our methodology divides a day into multiple timeslots and then, for each timeslot, derives relevant features such as the temporal shapes of the time series of the sensor data. In order to verify the advantage of our methodology, we collected a data set of smartphone usage from 25 participants for four weeks and obtained a license to a large-scale public data set constructed from 907 users over approximately nine months. The experimental results on the two data sets show that looking back to the beginning of the current day improves prediction accuracy by up to 16% and 7%, respectively, compared with the baseline and state-of-the-art methods.

In dimensional inspection using coordinate measuring machines (CMMs), the following issues are critical to achieve accurate inspection while minimizing the cost and time: 1) How can we select the sampling positions of the measurements so that we can get as much information from a limited number of samples as possible and 2) given the limited number of measurements, how can we assess the form error so that one can reliably decide whether the product is acceptable? To address these problems, we propose a wavelet-based model that takes advantage of the fact that the Lipschitz regularity holds for the CMM data. Under the framework of the proposed model, we derive the optimal sampling positions and propose a systematic procedure to estimate the form error given the limited number of sampled points. The proposed method is validated using both synthetic and real CMM data sets for straightness measurements. The comparison with other existing methods demonstrates the effectiveness of our method.

Learning and identifying events in network traffic is crucial for service providers to improve their mobility network performance. In fact, large special events attract cell phone users to relative small areas, which causes sudden surge in network traffic. To handle such increased load, it is necessary to measure the increased network traffic and quantify the impact of the events, so that relevant resources can be optimized to enhance the network capability. However, this problem is challenging due to several issues: (1) Multiple periodic temporal traffic patterns (i.e., nonhomogeneous process) even for normal traffic, (2) Irregularly distributed spatial neighbor information, (3) Different temporal patterns driven by different events even for spatial neighborhoods, (4) Large scale data set. This paper proposes a systematic event detection method that deals with the above problems. With the additivity property of Poisson process, we propose an algorithm to integrate spatial information by aggregating the behavior of temporal data under various areas. Markov Modulated Nonhomogeneous Poisson Process (MMNHPP) is employed to estimate the probability with which event happens, when and where the events take place, and assess the spatial and temporal impacts of the events. Localized events are then ranked globally for prioritizing more significant events. Synthetic data are generated to illustrate our procedure and validate the performance. An industrial example from a telecommunication company is also presented to show the effectiveness of the proposed method.

The objective of this study is to express the four types of personality of a robot based on Myers-Briggs Type Indicator by controlling the size, speed, and frequency of the gestures of a robot and to examine userpsila impressions of the robot by controlling the gesture design factors. The independent variables were three gesture design factors (speed, velocity, and frequency) with two levels each, producing eight gesture types. The eight gesture types were presented in two robot positions, a speaking robot and a listening robot. Participants watched a total of 16 robot gesture samples and evaluated each sample on the personality of the robot and their impressions. The evaluation results provided basic guidelines to express robot personalities through gestures, describing specific methods to control the speed, velocity, and frequency of a gesture. In addition, it was found that personality expression of a robot through gesture and the control of gesture design factors affect user impression of a robot.

We apply a fast 3D-SPIHT to compress hyperspectral images. The method uses a tree information matrix to determine whether a tree is significant or insignificant during encoding process. Experiments with AVIRIS data show that it is possible to save about 70% of processing time.