With the fast advancements in Internet of Things (IoT) technology and sensing infrastructure, a wide range of systems continues to generate a massive amount of data. Meanwhile, practical resource constraints such as limited bandwidth or processing capability restrict the full observability of data streams in real time. As a result, the practitioners often need to dynamically decide which data streams to observe given the resource constraints in order to quickly detect any system anomaly as soon as possible. In this article, we propose a reinforcement learning framework based on deep Q-learning and combine it with statistical process control (SPC) techniques to effectively monitor high-dimensional data streams when only partial observations are available at each acquisition time due to resource constraints. To the best of our knowledge, this is the first work that integrates deep reinforcement learning, which considers long-term rewards associated with the dynamic data sampling strategy, into SPC charts; this integration effectively addresses the challenge of resource constraints in monitoring high-dimensional data streams. Specifically, we construct a nonparametric monitoring statistic for each data stream and develop a reinforcement learning framework to automatically identify the most informative data streams for observation at each time epoch. The state space, action space, and rewards in the reinforcement learning framework are carefully designed and a Double Dueling Q-network is trained accordingly. Unlike existing methods, which rely on heuristic approaches to determine the sampling strategy, the proposed framework maximizes the long-term reward, thus leading to superior performance compared to existing benchmarks. Numerical simulations and a case study are thoroughly conducted, showing that the proposed method outperforms the state-of-the-art algorithms by significantly reducing detection delay. Note to Practitioners—This paper is motivated by the practical issue of online process monitoring and anomaly detection with resource constraints. In particular, due to resource constraints, practitioners can only select a subset of data streams to monitor at each time epoch. Thus, the central challenges are to dynamically choose which data streams to observe and to decide when to raise an alarm. Unlike the existing methodologies which are heuristic and only consider short-term rewards from the dynamic sampling, this paper proposes a novel reinforcement learning framework that allows practitioners to monitor high-dimensional heterogeneous data streams more efficiently by considering the long-term rewards associated with the dynamic data sampling strategy. Four main steps are involved in the proposed method: (i) construct a nonparametric local monitoring statistic for each data stream; (ii) train the Double Dueling Q-network offline according to the proposed deep reinforcement learning framework; (iii) at each time epoch of the online monitoring, determine the most informative data streams to observe based on the deep Q-network; and (iv) determine whether to raise an alarm in the system.

In today’s manufacturing industries, the development of sensor technology and Internet of Things has made real-time process monitoring of high-dimensional data increasingly vital. However, resource constraints, such as limited power, budget, and transmission capacity, often prevent access to full data streams in real time. This means that practitioners need to effectively monitor the process based on only partially observed data by dynamically deciding the sampling layout in real time. Another common challenge of process monitoring in practice is the lack of historical reference data, which can occur due to process/system upgrades or equipment replacements. To address these critical challenges, this paper proposes MASS (Monitoring with Adaptive Sampling under Self-starting scheme), a novel self-starting monitoring approach tailored to monitor high-dimensional data streams when only limited resources and historical reference data are available. Our monitoring framework is based on a quantile-based nonparametric CUSUM procedure with likelihood ratio-based statistics, and then the Thompson Sampling (TS) algorithm is adopted to handle partially observed data in the self-starting scenario. A key feature of our proposed method is its adaptive estimation of the out-of-control distribution and data quantiles, which ensures robust detection for various shifts in data streams with arbitrary and heterogeneous distributions, even in cases with limited reference data. The outperformance of the proposed method is demonstrated through simulation experiments and a real-world case study. Note to Practitioners—This paper is motivated by the critical challenges of online process monitoring when only limited resources (e.g., limited power availability, limited number of sensors, and limited transmission capacity) and limited historical in-control reference data are available. For example, consider a scenario where a newly established production system requires online monitoring across multiple data streams. In such cases, there is often a deficiency of reference data crucial for constructing a reliable control chart. Additionally, due to resource constraints, it is frequently infeasible to gather information from all streams associated with the production line at each epoch in real time. Unlike previous methods which require either a sufficient amount of reference data, or fully observable data streams, this paper proposes a novel monitoring and dynamic sampling scheme to effectively monitor partially observable data streams with only a small amount of reference data. To implement the methodology, it requires: (i) to initiate the process by estimating quantiles with a small amount of reference data, (ii) to determine which data streams to observe at each time epoch, (iii) to adaptively update the estimation of process parameters during online monitoring, and (iv) to construct a set of local and global statistics that can be used to quickly detect the system anomaly in real time. Numerical experiments and a real-world case study suggest that our proposed method efficiently leverages available data to reduce detection delays and enhance effectiveness against various shifts, in comparison to the benchmark methods.

Understanding the underlying component structure is crucial for multivariate signal analysis. Among all the techniques that try to learn the latent structure, independent component analysis (ICA) is one of the most important and popular methods, which aims to extract independent components from multivariate signals and enables further analysis. For example, in electroencephalogram (EEG) analysis, artifacts filtering and disease detection are conducted based on the independent components of the signals. One critical challenge in existing ICA approaches is that the component extraction accuracy may degrade when the available data of a unit are limited. To address this issue, this paper proposes a transfer learning-based ICA method by innovatively transferring component distribution from a source domain, so that accurate component extraction results can be achieved even when only limited data are available in the target domain. To the best of our knowledge, this is the first work that leverages transfer learning to improve ICA accuracy with limited available data. In particular, we first extract all the independent components from the source domain by maximizing the log-likelihood function with a Newton-like method on a smooth manifold. Then for the target domain, the component with the largest negentropy is extracted in each round. To effectively leverage the knowledge from the source domain and to prevent the negative transfer, we try to find a component in the source domain that matches the component we are extracting. The probability density function of the matched component will then be used to improve the component extraction accuracy if such matched component can be found; otherwise, no knowledge will be transferred. Numerical simulations and a case study with electrocardiogram (ECG) data are conducted, showing the effectiveness of the proposed method in transferring knowledge and reducing negative transfer. Note to Practitioners—This paper is motivated by the practical issue of conducting independent component analysis when data are limited to derive a reliable result. To address this challenge, we propose to transfer knowledge from a source domain to the target domain. Specifically, there are two fundamental questions involved: 1) what knowledge can be transferred from the source domain; and 2) how to minimize the negative transfer when no useful knowledge is available. Our novel idea is to transfer the component distributions and the negative transfer is largely reduced through a component matching step as a result. There are four main steps involved when implementing the proposed method: 1) solve all the independent components in the source domain; 2) extract the independent component in the target domain with the largest negentropy; 3) decide whether a matched component can be found from the source domain; and 4) re- estimate the current independent component in the target domain by leveraging the distribution information of the matched component. Step 2 to step 4 are repeated several times until all the independent components in the target domain are extracted or some termination condition is reached.

To avoid unexpected failures of engineering systems, sensors have been widely used to monitor the degradation process of the systems. A number of studies have been conducted to analyze the collected sensor signals and predict the failure time. However, the existing studies are usually restricted and cannot be adapted to different practical situations. In this paper, we propose a systematic method for degradation modeling and prognosis that can be widely applied in different scenarios. In particular, the proposed method is capable to handle one or multiple sensors, powerful to capture the nonlinear relations between sensor signals and the degradation process with few assumptions, generic to consider multiple failure modes, flexible to deal with unequally spaced sensor measurements or asynchronous signals, and easily understandable with little preprocessing required. The main idea is to predict the failure time of an in-service unit based on a subset of the nearest historical units, where features are extracted from each sensor to describe the progression of sensor signals and local linear regression models are constructed to establish the relation between failure time and the extracted features. The prediction variance is then used as the goodness-of-fit measure, based on which decision-level fusion and feature-level fusion are proposed to combine multiple sensors. A case study with two datasets on the degradation modeling of aircraft engines is conducted which shows satisfactory performance of the proposed method. Note to Practitioners—This paper aims at modeling the collected sensor signals to understand the degradation process of the monitored engineering systems and predict the failure time. The main idea is to measure the similarity of units and predict the failure time of an in-service unit based on a subset of the nearest historical units. The developed method is widely applicable in different practical situations such as multiple sensors, multiple failure modes, asynchronous signals, and missing data. Furthermore, the method requires little preprocessing. There are several steps involved for implementing the proposed method: 1) collecting the sensor signals for historical units and the in-service unit; 2) extracting features from each sensor signal; 3) constructing a local linear model to predict the failure time based on the extracted features, and obtaining the prediction variance on the in-service unit; and 4) combining the information of different sensors using the decision-level fusion or feature-level fusion, if each unit is monitored by multiple sensors.

As the main force of energy consumption, machines consume a huge amount of energy in some production systems. To operate the systems in an energy-efficient way, the efficiencies of the machines should be elaborately optimized so that the total energy consumption in the production lines is minimized while maintaining the required production rate. For this purpose, this article investigates the energy consumption optimization problem in Bernoulli serial lines having more than two machines. Specifically, this problem is first formulated as a nonlinear programming with a production rate constraint; due to difficulties in solving this constrained nonlinear programming, structural characteristics of the problem and properties of the objective function, which are inspired by the results and insights obtained in two-machine lines, are analyzed. Finally, based on the structural characteristics and properties explored, a solution methodology is developed to recursively solve the unique optimal solution of the energy consumption optimization problem. Extensive numerical experiments show that the solution provided by this method, which has the reversibility property and is better than the one provided by an existing method in the literature, is numerically optimal (note that the theoretically optimal solution is extremely hard to obtain if not totally impossible). Note to Practitioners-Machines consume a huge amount of energy in energy-intensive (i.e., high energy-consuming) production systems in, e.g., automobile, semiconductor, and steel companies. Due to economic, social, and environmental concerns, it is a high priority to consider ways to reduce the energy consumption in these production systems. In this article, the problem of minimizing the total energy consumption in Bernoulli serial lines while maintaining the required production rate is investigated. Based on the aggregation method developed for performance analysis of long lines and the optimization results obtained for two-machine lines, the energy consumption optimization problem for long Bernoulli serial lines is elaborately analyzed and a method is proposed to recursively solve its unique optimal solution (i.e., the optimal machine efficiencies). From the point of view of system operation, this research, which makes the system operate in the most energy-efficient manner, offers a new managerial paradigm for the production systems.