Estimating dependency structures from noisy multivariate variables is fundamentally important in many applications. Of particular importance in practice is anomaly localization, which is to compute a variable-wise anomaly score by comparing a target dependency structure to a reference structure. In this task, stably and accurately estimating the dependency structures is the key. First, we present an ℓ0-elastic net model for estimating sparse inverse covariance matrices. Then we introduce a framework for anomaly localization that utilizes both the ℓ0-elastic net model and a transfer learning model. Although ℓ0-constrained optimization is known to be challenging, we introduce a hard thresholding line-search algorithm to efficiently solve these graphical models. Using synthetic and real-world data sets, we demonstrate that the proposed ℓ0-based method systematically outperforms alternative methods in many use-cases.

In numerous machine learning applications, there is a preference for classifiers characterized by a polyhedral description, as they are intended for utilization within optimization frameworks or for interpretability purposes. Here, we present a structured classifier designed to cater to downstream decision-making tasks. The classification method is achieved through the process of partitioning the feature domain into clusters and encompassing each cluster within a polytope. We employ a combined approach that integrates semi-supervised k-means with SVM. This unified optimization framework enables the simultaneous generation of multiple polytopes. The central concept involves using a k-means-based clustering method for the clustering step, followed by the utilization of SVM to construct hyperplanes between each pair of clusters. Notably, the clustering process for each class considers classification loss as well as information from other classes when allocating sample points to clusters. We propose an algorithm to solve the integer program. Our numerical experiments demonstrate the competitiveness of the proposed method across a wide spectrum of datasets, exhibiting its efficacy in comparison to existing hyperplane-based classifiers and nonlinear classifiers.

Transformers have gained popularity in time series forecasting for their ability to capture long-sequence interactions. However, their memory and compute-intensive requirements pose a critical bottleneck for long-term forecasting, despite numerous advancements in compute-aware self-attention modules. To address this, we propose TSMixer, a lightweight neural architecture exclusively composed of multi-layer perceptron (MLP) modules. TSMixer is designed for multivariate forecasting and representation learning on patched time series, providing an efficient alternative to Transformers. Our model draws inspiration from the success of MLP-Mixer models in computer vision. We demonstrate the challenges involved in adapting Vision MLP-Mixer for time series and introduce empirically validated components to enhance accuracy. This includes a novel design paradigm of attaching online reconciliation heads to the MLP-Mixer backbone, for explicitly modeling the time-series properties such as hierarchy and channel-correlations. We also propose a Hybrid channel modeling approach to effectively handle noisy channel interactions and generalization across diverse datasets, a common challenge in existing patch channel-mixing methods. Additionally, a simple gated attention mechanism is introduced in the backbone to prioritize important features. By incorporating these lightweight components, we significantly enhance the learning capability of simple MLP structures, outperforming complex Transformer models with minimal computing usage. Moreover, TSMixer's modular design enables compatibility with both supervised and masked self-supervised learning methods, making it a promising building block for time-series Foundation Models. TSMixer outperforms state-of-the-art MLP and Transformer models in forecasting by a considerable margin of 8-60%. It also outperforms the latest strong benchmarks of Patch-Transformer models (by 1-2%) with a significant reduction in memory and runtime (2-3X).

AI Applications uses various data science tools such as Jupyter notebook to prescribe a series of steps, commonly referred as workflow, for building AI Solutions. The steps in workflow can be as simple as loading the data from remote storage, visualize the data for better understanding or conducting data quality study, or it can be as complex as generating features for modeling, best model discovery processes, etc. Clearly, different steps of the data science workflow has varying requirement of compute resources. Moreover, the execution of steps in workflow are Adhoc and Subjective. With wider availability of various Serverless technology, in this paper, we demonstrate a generalized framework that can be used to provide on demand scale out capability for the Data Science Workflow. In particular, we selected the most common AI operation, namely Automatic Model Selection, as an example to demonstrate benefits of serverless computing. We conducted a detailed experimental results using IBM Code Engine technology to validate the benefits of our proposed approach.

This poster paper presents an efficient design of deploying anomaly detection service using serverless technology. Our design is motivated by the fact that the workload originating from the service calls are adhoc and reserving the infrastructure upfront is not advisable. To address this, we utilized the emerging serverless platform for executing the incoming training request. Our extensive experimental analysis demonstrate the usefulness of the proposed idea.

Assets in the domain of Internet of Things (IoT) generate time-series data such as sensor readings and alerts. In addition, the assets have associated static data such as the make, model and other manufacturing information. The sensors in the asset components may have implicit relationships with each other, which are not interpretable without domain knowledge. Many problems exist which involve computation of relationships between sensors or subsystems in the asset components. Typically, the number of sensors in a real world asset may range anywhere from tens to thousands of sensors - and in this case, finding relationships between them becomes a highly computationally intensive task. In this paper, we study one such problem of anomaly detection in industrial data based on the functioning of the sensors and their interrelationships in both normal and abnormal conditions. We further demonstrate the issue of run-time and performance complexity in this problem, and present a speed-up strategy using Serverless Computing for parallelization, and demonstrate the usefulness of this method by comparing the speed-up achieved.

In this paper, we describe an overarching ML system with a simple programming interface that leverages existing AI and ML frameworks to make the task of model exploration easier. The proposed system introduces a new programming construct namely pipeline graph (a directed acyclic graph) consisting of multiple machine learning operations provided by different ML repositories. End user uses the pipeline graph as a common interface for modeling different ML tasks such as classification, regression, and timeseries prediction, while enabling efficient execution on different environments (Spark, Celery and Cloud). We further annotated the pipeline graph with a hyper-parameter grid and an option to try-out a wide range of optimization strategies (i.e., Random, Bayesian, Bandit, AutoLearn, etc). Given a large pre-defined pipeline graph along with its hyper-parameters, we provided a general-purpose, scalable and efficient pipeline-graph exploration technique to provide the automated solutions to a variety of ML tasks. We compare our automated approach to several state-of-the-art automated AI systems and find that we achieve performance comparable to the best results, while often producing simpler pipelines using off the shelf components. Our evaluation suite consists of experiments on 60+ classifications and regressions datasets.

Time series value forecasting using machine learning models utilizing time series features has recently got good attention of Time series analytics community. This paper proposes an automated feature learning mechanisms to filter out most useful features from hundreds of available features for time series prediction problems. The paper further proposes a novel mechanism to dynamically filter features that are most suitable for the given input time series data. With such mechanisms we create pipeline consisting of most useful features for given input data and increases the performance of the prediction model. Our proposed mechanism first, groups well known features for time series analysis, generates and assigns the features importance score using multiple scoring configurations. Once scores are assigned, features are filtered using a threshold that is derived using reference feature score and Critical Difference diagram. The filtered features are subsequently analyzed based on the characteristics of the input dataset. We show using experimental results that our approach of input data based dynamic feature selection improves the overall performance of machine learning models compared to the case where dynamic feature extraction is not applied prior to modeling.

The success of IoT depends on our ability to solve challenging problems which were previously infeasible. One of the most critical challenges in IoT space is the preventive maintenance in industrial manufacturing processes to maximize equipment availability and durability. Traditionally, preventive management only follows less cost-effective strategies, say time or usage based management. With large amount sensor data harvested from IoT, we can develop much more intelligent predictive maintenance based on accurate machinery failure prediction. In this paper, we develop a framework named DoM (Doctor for Machines) to produce the best predictive model for several oil and gas industry engagements. Our framework is built in the form of pipeline that allows us to generate multiple models simultaneously with parallel computing. We configure the modeling process by assigned different machine learning tasks as such sampling, feature extraction, modeling, and post-processing into the pipeline. The pipeline forms a machine learning graph workflow. With the automation of execution all the tasks in the workflow, the client can easily choose a best predictive model fitting their failure tolerance. We have successfully applied DoM to six data sets, and identified valuables insights on the best practices of creating predictive model. Our clients have successfully achieved financial benefits by applying DoM to develop predictive maintenance schemes to their expensive equipment.

Deep neural networks (DNNs) give state-of-the-art accuracy in many tasks, but they can require large amounts of memory storage, energy consumption, and long inference times. Modern DNNs can have hundreds of million parameters, which make it difficult for DNNs to be deployed in some applications with low-resource environments. Pruning redundant connections without sacrificing accuracy is one of popular approaches to overcome these limitations. We propose two l0-constrained optimization models for pruning deep neural networks layer-by-layer. The first model is devoted to a general activation function, while the second one is specifically for a ReLU. We introduce an efficient cutting plane algorithm to solve the latter to optimality. Our experiments show that the proposed approach achieves competitive compression rates over several state-of-the-art baseline methods.

Fueled with growth in the fields of Internet of Things (IoT) and Big Data, data has become one of the most valuable assets in today's world. While we are leveraging this data for analyzing complex systems using machine learning and deep learning, a considerable amount of time and effort is spent on addressing data quality issues. If undetected, data quality issues can cause large deviations in the analysis, misleading data scientists. To ease the effort of identifying and addressing data quality challenges, we introduce DQA, a scalable, automated and interactive data quality advisor. In this paper, we describe the DQA framework, provide detailed description of its components and the benefits of integrating it in a data science process. We propose a programmatic approach for implementing the data quality framework which automatically generates dynamic executable graphs for performing data validations fine-tuned for a given dataset. We discuss the use of DQA to build a library of validation checks common to many applications. We provide insight into how DQA addresses many persistence and usability issues which currently make data cleaning a laborious task for data scientists. Finally, we provide a case study of how DQA is implemented in a realworld system and describe the benefits realized.

Discovering the underlying dynamics leading up to an industrial asset failure is an important problem to be solved for successful development of Predictive Maintenance techniques. Existing work has largely focused on building complex ML/AI models for developing Predictive Maintenance solution patterns, but has largely avoided developing methods to explain the underlying failure dynamics. In this paper, we use an old but significantly improved change-pattern based technique to analyze IoT sensor data and failure information to generate useful and interpretable failure-centric insight. We discuss a solution pattern that we call ChieF, which when applied on multi-variate time series datasets, discover the leading failure indicators, generate associative patterns among multiple features, and output temporal dynamics of changes. Experimental analysis of ChieF on four datasets uncovers insights that may be valuable for predictive maintenance.

This paper proposes a new framework for anomaly detection when collectively monitoring many complex systems. The prerequisite for condition-based monitoring in industrial applications is the capability of (1) capturing multiple operational states, (2) managing many similar but different assets, and (3) providing insights into the internal relationship of the variables. To meet these criteria, we propose a multi-task learning approach based on a sparse mixture of sparse Gaussian graphical models (GGMs). Unlike existing fused- and group-lasso-based approaches, each task is represented by a sparse mixture of sparse GGMs, and can handle multi-modalities. We develop a variational inference algorithm combined with a novel sparse mixture weight selection algorithm. To handle issues in the conventional automatic relevance determination (ARD) approach, we propose a new ℓ0-regularized formulation that has guaranteed sparsity in mixture weights. We show that our framework eliminates well-known issues of numerical instability in the iterative procedure of mixture model learning. We also show better performance in anomaly detection tasks on real-world data sets. To the best of our knowledge, this is the first proposal of multi-task GGM learning allowing multi-modal distributions.

We propose a new approach to anomaly detection from multivariate noisy sensor data. We address two major challenges: To provide variable-wise diagnostic information and to automatically handle multiple operational modes. Our task is a practical extension of traditional outlier detection, which is to compute a single scalar for each sample. To consistently define the variable-wise anomaly score, we leverage a predictive conditional distribution. We then introduce a mixture of Gaussian Markov random field and its Bayesian inference, resulting in a sparse mixture of sparse graphical models. Our anomaly detection method is capable of automatically handling multiple operational modes while removing unwanted nuisance variables. We demonstrate the utility of our approach using real equipment data from the oil industry.

Food safety is an important health issue in Singapore as the number of food poisoning cases have increased significantly over the past few decades. The National Environment Agency of Singapore (NEA) is the primary government agency responsible for monitoring and mitigating the food safety risks. In an effort to pro-actively monitor emerging food safety issues and to stay abreast with developments related to food safety in the world, NEA tracks the World Wide Web as a source of news feeds to identify food safety related articles. However, such information gathering is a difficult and time consuming process due to information overload. In this paper, we present FoodSIS, a system for end-to-end web information gathering for food safety. FoodSIS improves efficiency of such focused information gathering process with the use of machine learning techniques to identify and rank relevant content. We discuss the challenges in building such a system and describe how thoughtful system design and recent advances in machine learning provide a framework that synthesizes interactive learning with classification to provide a system that is used in daily operations. We conduct experiments and demonstrate that our classification approach results in improving the efficiency by average 35% compared to a conventional approach and the ranking approach leads to average 16% improvement in elevating the ranks of relevant articles.

Learning temporal graph structures from time series data reveals important dependency relationships between current observations and histories. Most previous work focuses on learning and predicting with static temporal graphs only. However, in many applications such as mechanical systems and biology systems, the temporal dependencies might change over time. In this paper, we develop a dynamic temporal graphical models based on hidden Markov model regression and lasso-type algorithms. Our method is able to integrate two usually separate tasks, i.e. inferring underlying states and learning temporal graphs, in one unified model. The output temporal graphs provide better understanding about complex systems, i.e. how their dependency graphs evolve over time, and achieve more accurate predictions. We examine our model on two synthetic datasets as well as a real application dataset for monitoring oil-production equipment to capture different stages of the system, and achieve promising results.

The development of an aggregate view of the procurement spend across an enterprise using transactional data is increasingly becoming a very important and strategic activity. Not only does it provide a complete and accurate picture of what the enterprise is buying and from whom, it also allows it to consolidate suppliers, as well as negotiate better prices. The importance, as well as the complexity, of this cleansing exercise is further magnified by the increasing popularity of Business Transformation Outsourcing (BTO) wherein enterprises are turning over non-core activities, such as indirect procurement, to third parties, who now need to develop an integrated view of spend across multiple enterprises in order to optimize procurement and generate maximum savings. However, the creation of such an integrated view of procurement spend requires the creation of a homogeneous data repository from disparate (heterogeneous) data sources across various geographic and functional organizations throughout the enterprise(s). Such repositories get transactional data from various sources such as invoices, purchase orders, account ledgers. As such, the transactions are not cross-indexed, refer to the same suppliers by different names, and use different ways of representing information about the same commodities. Before an aggregated spend view can be developed, this data needs to be cleansed, primarily to normalize the supplier names and correctly map each transaction to the appropriate commodity code. Commodity mapping, in particular, is made more difficult by the fact that it has to be done on the basis of unstructured text descriptions found in the various data sources. We describe an on-demand system to automatically perform this cleansing activity using techniques from information retrieval and machine learning. Built on standard integration and application infrastructure software, this system provides enterprises with a fast, reliable, accurate and on-demand way of cleansing transactional data and generating an integrated view of spend. This system is currently in the process of being deployed by IBM for use in its BTO practice.

Part numbers are widely used within an enterprise throughout the manufacturing process. The point of entry of such part numbers into this process is normally via a Bill of Materials, or BOM, sent by a contact manufacturer or supplier. Each line of the BOM provides information about one part such as the supplier part number, the BOM receiver's corresponding internal part number, an unstructured textual part description, the supplier name, etc. However, in a substantial number of cases, the BOM receiver's internal part number is absent. Hence, before this part can be incorporated into the receiver's manufacturing process, it has to be mapped to an internal part (of the BOM receiver) based on the information of the part in the BOM. Historically, this mapping process has been done manually which is a highly time-consuming, labor intensive and error-prone process. This paper describes a system for automating the mapping of BOM part numbers. The system uses a two step modeling and mapping approach. First, the system uses historical BOM data, receiver's part specifications data and receiver's part taxonomic data along with domain knowledge to automatically learn classification models for mapping a given BOM part description to successively lower levels of the receiver's part taxonomy to reduce the set of potential internal parts to which the BOM part could map to. Then, information about various part parameters is extracted from the BOM part description and compared to the specifications data of the potential internal parts to choose the final mapped internal part. Mappings done by the system are very accurate, and the system is currently being deployed within IBM for mapping BOMs received by the corporate procurement/manufacturing divisions.

The process-driven composition of Web services is emerging as a promising approach to integrate business applications within and across organizational boundaries. In this approach, individual Web services are federated into composite Web services whose business logic is expressed as a process model. The tasks of this process model are essentially invocations to functionalities offered by the underlying component services. Usually, several component services are able to execute a given task, although with different levels of pricing and quality. In this paper, we advocate that the selection of component services should be carried out during the execution of a composite service, rather than at design-time. In addition, this selection should consider multiple criteria (e.g., price, duration, reliability), and it should take into account global constraints and preferences set by the user (e.g., budget constraints). Accordingly, the paper proposes a global planning approach to optimally select component services during the execution of a composite service. Service selection is formulated as an optimization problem which can be solved using efficient linear programming methods. Experimental results show that this global planning approach outperforms approaches in which the component services are selected individually for each task in a composite service.