Given the scarcity of anomalies in real-world applications, the majority of literature has been focusing on modeling normality. The learned representations enable anomaly detection as the normality model is trained to capture certain key underlying data regularities under normal circumstances. In practical settings, particularly industrial time series anomaly detection, we often encounter situations where a large amount of normal operation data is available along with a small number of anomaly events collected over time. This practical situation calls for methodologies to leverage these small number of anomaly events to create a better anomaly detector. In this paper, we introduce two methodologies to address the needs of this practical situation and compared them with recently developed state of the art techniques. Our proposed methods anchor on representative learning of normal operation with autoregressive (AR) model along with loss components to encourage representations that separate normal versus few positive examples. We applied the proposed methods to two industrial anomaly detection datasets and demonstrated effective performance in comparison with approaches from literature. Our study also points out additional challenges with adopting such methods in practical applications.

Clustering is a challenging problem when many features are irrelevant to separate clusters. Also, different clusters may relate to various feature subsets. This work proposes a deep clustering algorithm that localizes the search for instance clusters and their relevant features. The relevant features of each cluster are defined as those with high associations (dependency) within that cluster. Given the number of clusters$K$, we formulate the problem as$K$. -parallel auto-reconstructive learning, where low-rank graph learning, rooted in graph Laplacian theory, is used to explore the unknown feature associations of each cluster. The model performs automatic feature weighting on residuals to minimize loss from the corresponding cluster. Through such design, different feature subsets can be learned to calculate the loss from different clusters. Subsequently, the associations between features and clusters can be acquired, and better clustering result can be achieved. Moreover, the associated features of each cluster can be used to interpret the clustering patterns.

Identifying the faulty class of multivariate time series is crucial for today's flight data analysis. However, most of the existing time series classification methods suffer from imbalanced data and lack of model interpretability, especially on flight data of which faulty events are usually uncommon with a limited amount of data. Here, we present a neural network classification model for imbalanced multivariate time series by leveraging the information learned from normal class, which can also learn the nonlinear Granger causality for each class, so that we can pinpoint how time series classes differ from each other. Experiments on simulated data and real flight data shows that this model can achieve high accuracy of identifying anomalous flights.

Product development in modern enterprises usually involves collaboration among designers, suppliers, contract manufacturers to achieve efficiency and rapid response to changing markets. Product-development cost, lead time, and reliability are very critical elements in addition to functional features. However, it becomes increasingly challenging to obtain an optimal decision with respect to these multiple criteria as the number of involved entities increases in modern product developments. There is a clear need for planning tools to support effective decision making in this domain. The availability of efficient and accurate multiobjective optimization (MOO) algorithms becomes critical in such a decision support tool. This paper poses the product development as a multiobjective assignment problem in the context of printed circuit board assembly (PCBA) industry. We describe a new class of MOO algorithm based on the principles of differential evolution (DE). The multiobjective DE (MODE) algorithm is shown to approach Pareto-optimal solutions in a wide class of problems with better performance than the nondominated sorting genetic algorithm II from the literature, providing a practical tool for product-development decision support. A decision support system based on the object-oriented design methodology is described in this paper with the MODE as the core search engine. Experimental study of this decision support system is conducted using two real-world PCBA designs. We demonstrate the effectiveness of this proposed MODE algorithm and some use cases of such decision support system on facilitating decision makers' tradeoff analysis.

Jet engine gas path fault diagnosis is not only important in modern condition-based maintenance of aircraft engines, but also a challenging classification problem. Exploring more advanced classification techniques for achieving improved classification performance for gas path fault diagnosis, therefore, has been increasingly active in recent years in PHM community. To that end, in this paper, we apply a recently developed dynamic fusion scheme to gas path fault diagnosis. Through designing a real-world gas path fault diagnostic system, we demonstrate that dynamic fusion of multiple classifiers can be effective in improving classification performance of gas path diagnosis.

Locomotives are complex electromechanical systems. Continuously monitoring the health state of locomotives is critical in modern cost-effective maintenance strategy. A typical locomotive is equipped with the capability to generate fault messages or incidents based on logical rules in the control system. In the mean time, sensor readings and operational state variables are also collected. The goal is to detect faults early to provide lead-time for maintenance actions and trip planning based on the collected fault log and parametric data. In this paper, we present a model-based anomaly detection strategy. In this method, the inputs-outputs relationship of a locomotive subsystem is modeled using a neural network model based on normal operational data. The residuals between measurements and model outputs are calculated. A reasoning module based on these multiple residuals is used to generate an overall health indicator of the subsystem at each instance of times, which is further used to determine whether the subsystem is abnormal. Statistical testing, Gaussian mixture model and support vector machine are used to generate this healthy index and their performances are compared. We demonstrate the effectiveness of the anomaly detection strategy using real-world operational data from locomotives.

Fusing the outputs from an ensemble of models in an effective way can often boost overall model accuracy. This paper presents a novel method, called locally weighted fusion, which aggregates the results of multiple predictive models based on local accuracy measures of these models in the neighborhood of the probe point for which we want to make a prediction. While we demonstrate the method in the context of multiple neural network models, the concepts may be applied to other predictive techniques as well. This fusion method is applied to develop highly accurate models for emissions, efficiency, and load prediction in a complex real-world power plant. The locally weighted fusion method boosts the predictive performance by 20-40% over the baseline single model approach for the various prediction targets. Relative to this approach, fusion strategies which apply averaging or globally weighting only produce a 2-6% performance boost over the baseline.

This paper reports a mathematical modeling and convergence analysis of a continuous multi-objective differential evolution (C-MODE) algorithm that is proposed very recently. This C-MODE is studied in the context of global random search. The convergence of the population to the Pareto optimal solutions with probability one is developed. In order to facilitate the understanding of the C-MODE operators in a continuous space, a mathematical analysis of the operators is conducted based upon a Gaussian distributed initial population. A set of guidelines is derived for the parameter setting of the C-MODE based on the theoretical results from the mathematical analysis. A simulation analysis on a specific numerical example is conducted to validate the mathematical analytical results and parameter-setting guidelines. The performance comparison based on a suite of complex benchmark functions also demonstrates the merits of such parameter-setting guidelines

The revival of multi-objective optimization (MOO) is mostly due to the recent development of evolutionary multi-objective optimization that allows the generation of the whole Pareto optimal front. Several evolutionary algorithms have been developed for this purpose. This paper focuses on the recent development of differential evolution (DE) algorithms for the multi-objective optimization purposes. Although there are a few other papers on the extension of DE concept to the MOO domain, this paper is intended to provide an overall picture of one specific multi-objective differential evolution (MODE) algorithm. In the MODE, the DE concept for the continuous single-objective optimization is extended to MOO for both continuous and discrete problems (C-MODE and D-MODE, respectively). The MODE is modeled in the context of Markov framework and global random search. Convergence properties are developed for both C-MODE and D-MODE. In particular, a set of parameter-setting guidelines for the C-MODE is derived based on the mathematical analysis. An application of the D-MODE to the planning of design, supply, and manufacturing resources in product development is also reported in this paper

In recent years, multi-objective evolutionary algorithms (MOEA) have generated a large research interest. MOEA's attraction stems from their ability to find a set of Pareto solutions rather than any single, aggregated optimal solution for a multi-objective problem. As for single-objective evolutionary algorithms (SOEA), multi-objective evolutionary algorithms also require parameter tuning to achieve desirable performance. In the literature we can find fuzzy logic controllers (FLC's) applied to online parameter control for SOEA. In this paper, we propose to use a FLC to dynamically adjust the parameters of a particular multi-objective differential evolution (MODE) algorithm. The fuzzy logic controlled multi-objective differential evolution (FLC-MODE) is applied to a suite of benchmark functions. Its results are compared to those obtained by using MODE with constant parameter settings. We show that the FLC-MODE obtains better results in 80% of the testing examples. Given that the benchmarks were synthetic test functions, we designed the FLC using only our understanding of the working mechanism of the MODE, without incorporating any additional problem-specific knowledge. When addressing real-world applications, we expect the FLC to be an excellent way for representing and leveraging their associated heuristic knowledge

Evolutionary multiobjective optimization (EMOO) finds a set of Pareto solutions rather than any single aggregated optimal solution for a multiobjective problem. The purpose is to describe a newly developed evolutionary approach-Pareto-based multiobjective differential evolution (MODE). The concept of differential evolution, which is well-known in the continuous single-objective domain for its fast convergence and adaptive parameter setting, is extended to the multiobjective problem domain. A Pareto-based approach is proposed to implement the differential vectors. A set of benchmark test functions is used to validate this new approach. We compare the computational results with those obtained in the literature, specifically by strength Pareto evolutionary algorithm (SPEA). It is shown that this new approach tends to be more effective in finding the Pareto front in the sense of accuracy and approximate representation of the real Pareto front with comparable efficiency.

Agility is important to modern enterprises. The effective coordination of large numbers of potential suppliers and manufacturer, demands a scientific methodology rather than just practical experience to make decisions on supply manufacturing planning problems. Particularly in cases where multiple decision objectives are important to process planning, empirical decisions are insufficient. This paper introduces formal methods to solve such multi-objective decision problems involved in general supply manufacturing planning, and specifically describes the extension of differential evolution methods to discrete problem domains. An enterprise planning problem with two objectives-cycle time and cost is used as a principal example. Such multi-objective optimization problems usually are very large and nonlinear. In this paper, the concept of differential evolution, which is well-known in single-objective continuous domain for its fast convergence and adaptive parameter setting, is extended to the discrete domain by introducing greedy probability, mutation probability, and crossover probability. Moreover, this concept is extended to discrete multi-objective optimization problem. The proposed discrete multi-objective differential evolution, or D-MODE algorithm is applied to obtain Pareto solutions of this general planning problem. A practical example in the electronics industry is used as an illustrative example to demonstrate the effectiveness of the proposed D-MODE.