The detection of anomalous behavior of an engineered system or its components is an important task for enhancing reliability, safety, and efficiency across various engineering applications. However, designing an accurate anomaly detector can be very challenging in settings where anomalous labels are sparse or, in the worst case, missing in the training data. To mitigate this issue of the lack of anomalous labels in the domain of interest, existing approaches use transfer learning by leveraging information from anomalous samples in a closely related source domain. Although previous studies have shown good results from applying transfer learning, they do not specifically address the issue of high false-positive rates from such transfer. High false-positive rates can arise from misleading information present in uninformative/irrelevant features. Inspired by this observation, this paper focuses on identifying key input features, termed as such, due to their strong predictability in anomaly detection. A transfer learning approach is introduced that leverages the optimal$f_{\beta } $score for key feature estimation. This approach involves finding a weight vector to amplify key features and attenuate uninformative inputs during prediction. We demonstrate the capabilities of our proposed anomaly detection method as a quality check for newly manufactured automotive transmissions. Given the use of frequency domain order-based features in our use case, our proposed method is also easily extensible to the anomaly detection of other rotating machinery. Based on our findings we also find that our proposed anomaly detection algorithm, utilizing precise data-driven features, outperforms detectors based on experience/heuristics-based features currently used in automotive engineering applications. More importantly, our proposed framework can work with any downstream unsupervised anomaly detection algorithm, allowing us to freely choose the best algorithm for the anomaly detection task on hand. Note to Practitioners—Unlike legacy systems that have rich data sets for both healthy and anomalous behaviors over a broad spectrum of operating states, newly designed system or design variants of existing systems have little or no performance data. This makes the task of identifying indicators of anomalous behaviors very challenging for systems without legacy information. This work introduces a machine learning framework that uses label information from related, well-studied systems, to improve anomaly detection outcomes in a new or variant (but related) system. We demonstrate the effectiveness of our transfer approach through the End of Line (EoL) anomaly detection for quality checks on new design variants of an existing automotive transmission. Inspired by feature selection techniques, our proposed framework is easy to use and offers excellent interpretability, which is crucial for achieving a zero-failure system design.

We propose a memory-augmented deep learning model for semisupervised anomaly detection (AD). While many traditional AD methods focus on modeling the distribution of normal data, additional constraints in the modeling process are needed to distinguish between normal and abnormal data. The proposed model, named memory augmented generative adversarial networks (MEMGAN), is coupled with external memory units through attentional operations. One property of MEMGAN in the latent space is such that encoded normal data are expected to reside in the convex hull of the memory units, while the abnormal ones are separated outside. This property makes the AD process of MEMGAN more robust and reliable. Experiments on AD datasets adapted from MVTec, MNIST, CIFAR10, and Arrhythmia demonstrate that MEMGAN notably improves over previous AD models. We also find that the decoded memory units in MEMGAN are more diverse and interpretable than those in previous memory-augmented models.

Language agnostic and semantic-language information isolation is an emerging research direction for multilingual representations models.We explore this problem from a novel angle of geometric algebra and semantic space.A simple but highly effective method "Language Information Removal (LIR)" factors out language identity information from semantic related components in multilingual representations pre-trained on multi-monolingual data.A post-training and model-agnostic method, LIR only uses simple linear operations, e.g.matrix factorization and orthogonal projection.LIR reveals that for weak-alignment multilingual systems, the principal components of semantic spaces primarily encodes language identity information.We first evaluate the LIR on a cross-lingual question answer retrieval task (LAReQA), which requires the strong alignment for the multilingual embedding space.Experiment shows that LIR is highly effectively on this task, yielding almost 100% relative improvement in MAP for weakalignment models.We then evaluate the LIR on Amazon Reviews and XEVAL dataset, with the observation that removing language information is able to improve the cross-lingual transfer performance.

This paper presents a novel training method, Conditional Masked Language Modeling (CMLM), to effectively learn sentence representations on large scale unlabeled corpora.CMLM integrates sentence representation learning into MLM training by conditioning on the encoded vectors of adjacent sentences.Our English CMLM model achieves state-ofthe-art performance on SentEval (Conneau and Kiela, 2018), even outperforming models learned using supervised signals.As a fully unsupervised learning method, CMLM can be conveniently extended to a broad range of languages and domains.We find that a multilingual CMLM model co-trained with bitext retrieval (BR) and natural language inference (NLI) tasks outperforms the previous state-of-the-art multilingual models by a large margin, e.g.10% improvement upon baseline models on cross-lingual semantic search.We explore the same language bias of the learned representations, and propose a simple, post-training and model agnostic approach to remove the language identifying information from the representation while still retaining sentence semantics.

In this paper, we investigate algorithms for anomaly detection. Previous anomaly detection methods focus on modeling the distribution of non-anomalous data provided during training. However, this does not necessarily ensure the correct detection of anomalous data. We propose a new Regularized Cycle Consistent Generative Adversarial Network (RCGAN) in which deep neural networks are adversarially trained to better recognize anomalous samples. This approach is based on leveraging a penalty distribution with a new definition of the loss function and novel use of discriminator networks. It is based on a solid mathematical foundation, and proofs show that our approach has stronger guarantees for detecting anomalous examples compared to the current state-of-the-art. Experimental results on both real-world and synthetic data show that our model leads to significant and consistent improvements on previous anomaly detection benchmarks. Notably, RCGAN improves on the state-of-the-art on the KDDCUP, Arrhythmia, Thyroid, Musk and CIFAR10 datasets.

Due to the ubiquitous use of embeddings as input representations for a wide range of natural language tasks, imputation of embeddings for rare and unseen words is a critical problem in language processing.Embedding imputation involves learning representations for rare or unseen words during the training of an embedding model, often in a post-hoc manner.In this paper, we propose an approach for embedding imputation which uses grounded information in the form of a knowledge graph.This is in contrast to existing approaches which typically make use of vector space properties or subword information.We propose an online method to construct a graph from grounded information and design an algorithm to map from the resulting graphical structure to the space of the pre-trained embeddings.Finally, we evaluate our approach on a range of rare and unseen word tasks across various domains and show that our model can learn better representations.For example, on the Card-660 task our method improves Pearson's and Spearman's correlation coefficients upon the stateof-the-art by 11% and 17.8% respectively using GloVe embeddings.

Summary High performance fast multipole method is crucial for the numerical simulation of many physical problems. In a previous study, we have shown that task‐based fast multipole method provides the flexibility required to process a wide spectrum of particle distributions efficiently on multicore architectures. In this paper, we now show how such an approach can be extended to fully exploit heterogeneous platforms. For that, we design highly tuned graphics processing unit (GPU) versions of the two dominant operators P2P and M2L) as well as a scheduling strategy that dynamically decides which proportion of subsequent tasks is processed on regular CPU cores and on GPU accelerators. We assess our method with the StarPU runtime system for executing the resulting task flow on an Intel X5650 Nehalem multicore processor possibly enhanced with one, two, or three Nvidia Fermi M2070 or M2090 GPUs (Santa Clara, CA, USA). A detailed experimental study on two 30 million particle distributions (a cube and an ellipsoid) shows that the resulting software consistently achieves high performance across architectures. Copyright © 2015 John Wiley & Sons, Ltd.

Molecular modeling is a field that traditionally has large computational costs. Until recently, most simulation techniques relied on long trajectories, which inherently have poor scalability. A new class of methods is proposed that requires only a large number of short calculations, and for which minimal communication between computer nodes is required. We considered one of the more accurate variants called Accelerated Weighted Ensemble Dynamics (AWE) and for which distributed computing can be made efficient. We implemented AWE using the Work Queue framework for task management and applied it to an all atom protein model (Fip35 WW domain). We can run with excellent scalability by simultaneously utilizing heterogeneous resources from multiple computing platforms such as clouds (Amazon EC2, Microsoft Azure), dedicated clusters, grids, on multiple architectures (CPU/GPU, 32/64bit), and in a dynamic environment in which processes are regularly added or removed from the pool. This has allowed us to achieve an aggregate sampling rate of over 500 ns/hour. As a comparison, a single process typically achieves 0.1 ns/hour.

Heterogeneous computers with processors and accelerators are becoming widespread in scientific computing. However, it is difficult to program hybrid architectures and there is no commonly accepted programming model. Ideally, applications should be written in a way that is portable to many platforms, but providing this portability for general programs is a hard problem.

Commercial graphics processors (GPUs) have high compute capacity at very low cost, which makes them attractive for general purpose scientic computing. In this poster we show how graphics processors can be used for N-body simulations to obtain large improvements in performance over current generation CPUs. We have developed a highly optimized algorithm for performing the O(N^2) force calculations that constitute the major part of stellar and molecular dynamics simulations. In the calculations, we achieve sustained performance of nearly 100 GFlops on an ATI X1900XTX. The performance on GPUs 25x an Intel Pentium4, and 2x specialized hardware such as GRAPE-6A, but at a fraction of the cost. Furthermore, the wide availability of GPUs has signicant implications for cluster computing and distributed computing efforts like [email protected]

The Merrimac supercomputer uses stream processors and a high-radix network to achieve high performance at low cost and low power. The stream architecture matches the capabilities of modem semiconductor technology with compute-intensive parallel applications. We present a detailed case study of porting the GROMACS molecular-dynamics force calculation to Merrimac. The characteristics of the architecture which stress locality, parallelism, and decoupling of memory operations and computation, allow for high performance of compiler optimized code. The rich set of hardware memory operations and the ample computation bandwidth of the Merrimac processor present a wide range of algorithmic trade-offs and optimizations which may be generalized to several scientific computing domains. We use a cycle-accurate hardware simulator to analyze the performance bottlenecks of the various implementations and to measure application run-time. A comparison with the highly optimized GROMACS code, tuned for an Intel Pentium 4, confirms Merrimac’s potential to deliver high performance.