To deliver business value, most data-driven enterprises and applications require data to be extracted and merged from otherwise siloed data storage platforms. FDC Cache has been designed and developed to enable the fusion and caching of data drawn from multiple small and/or Big Data stores. This capability executes a sequence of queries, wherein the results from one query may be used to constrain subsequent queries. The results of each query are linked with results from previous queries, incrementally building a cache of semantically linked data that can be used to support multiple independent data requests. FDC Cache uses Semantic Web technologies, and knowledge graphs in particular, to describe the relevant data and relationships in a computable model. This enables applications to reason over the graph, for example to dynamically retrieve targeted subsets of data comprised of previously disparate information. We have successfully applied FDC Cache to two distinct industrial use cases: (i) merging data across multiple sources to assemble information about current parts in a gas turbine, and (ii) dynamically aligning siloed data from electric grid transmission and distribution networks to an industry-standard common model, in which the cache creation time has been shown to scale sub-linearly with the number of data elements. FDC Cache has been open-sourced as part of the GE-developed open source Semantics Toolkit.

This work considers the canonical polyadic decomposition (CPD) of tensors using proximally regularized sketched alternating least squares algorithms. First, it establishes a sublinear rate of convergence for proximally regularized sketched CPD algorithms under two natural conditions that are known to be satisfied by many popular forms of sketching. Second, it demonstrates that the iterative nature of CPD algorithms can be exploited algorithmically to choose more performant sketching rates. This is accomplished by introducing CPD-MWU, a proximally-regularized sketched alternating least squares algorithm that adaptively selects the sketching rate at each iteration. On both synthetic and real data we observe that for noisy tensors CPD-MWU produces decompositions of comparable accuracy to the standard CPD decomposition in less time, often half the time; for ill-conditioned tensors, given the same time budget, CPD-MWU produces decompositions with an order-of-magnitude lower relative error. For a representative real-world dataset CPD-MWU produces residual errors on average 20% lower than CPRAND-MIX and 44% lower than SPALS, two recent sketched CPD algorithms.

Additive technologies are expected to revolutionize manufacturing across almost every industry, but there is a sizable gap between the current state of the technology and the maturity required for it to achieve widespread adoption. Advanced analytics are required to improve the reliability and repeatability of additive manufacturing, and those analytics require data. Large volumes of multimodal data are generated and used throughout the additive manufacturing lifecycle, from material design to part design and simulation, part printing to post-processing and inspection. To capture and link that diverse Big Data together, we have designed and developed a federated multimodal Big Data storage and analytics platform comprised of three tiers-a distributed polyglot data storage and analysis tier with different repositories for different data structures, a metadata knowledge graph tier for modeling the data and their relationships across the various repositories, and a user interface tier for visualizing, exploring and invoking analytics on the data. The platform has been used to integrate a collection of previously disparate steps to optimize the process parameters used to build a new additive material, enabling materials scientists and other non-software experts to capture, visualize and analyze the requisite data through a single user interface.

Canonical Polyadic Decomposition (CPD) is a powerful technique for uncovering multilinear relationships in tensors. Current research in scalable CPD has focused on designing efficient decomposition algorithms for large sparse tensors that arise in machine learning and data mining applications. This work addresses the complementary need for efficient decomposition algorithms for large dense tensors that arise in signal processing applications. Such tensors are often highly skewed, with one mode (e.g., time) orders of magnitude larger than the others. We present an algorithm appropriate for MapReduce settings that uses both regularization and sketching to efficiently operate on such tensors. We have open-sourced an Apache Spark implementation of the algorithm and evaluate it on synthetic and real datasets to characterize the trade-offs in runtime and accuracy when using different types and combinations of regularization and sketching. We observe that a combination of random entry sketching plus Tikhonov regularization works best independent of the type or level of noise in the tensor. Similarly, we find that random entry sketching plus proximal regularization works best for ill-conditioned tensors. Further experiments demonstrate that the runtime scales sublinearly with the tensor size and highly sublinearly with the tensor rank. The use of regularization and sketching results in runtimes that are factors of 42-112× faster than those of the previous state-of-the-art MapReduce CPD implementation for large dense, skewed tensors, while having a negligible impact on the accuracy of the decompositions.

The recent successes of commercial cognitive and AI applications have cast a spotlight on knowledge graphs and the benefits of consuming structured semantic data. Today, knowledge graphs are ubiquitous to the extent that organizations often view them as a “single source of truth” for all of their data and other digital artifacts. In most organizations, however, Big Data comes in many different forms including time series, images, and unstructured text, which often are not suitable for efficient storage within a knowledge graph. This paper presents the Semantics Toolkit (SemTK), a framework that enables access to polyglot-persistent Big Data stores while giving the appearance that all data is fully captured within a knowledge graph. SemTK allows data to be stored across multiple storage platforms (e.g., Big Data stores such as Hadoop, graph databases, and semantic triple stores) - with the best-suited platform adopted for each data type - while maintaining a single logical interface and point of access, thereby giving users a knowledge-driven veneer across their data. We describe the ease of use and benefits of constructing and querying polystore knowledge graphs with SemTK via four industrial use cases at GE.

Most organizations are becoming increasingly data-driven, often processing data from many different sources to enable critical business operations. Beyond the well-addressed challenge of storing and processing large volumes of data, financial institutions in particular are increasingly subject to federal regulations requiring high levels of accountability for the accuracy and lineage of this data. For companies like GE Capital, which maintain data across a globally interconnected network of thousands of systems, it is becoming increasingly challenging to capture an accurate understanding of the data flowing between those systems. To address this problem, we designed and developed a concept lineage tool allowing organizational data flows to be modeled, visualized and interactively explored. This tool has novel features that allow a data flow network to be contextualized in terms of business-specific metadata such as theconcept,business, andproductfor which it applies. Key analysis features have been implemented, including the ability to trace the origination of particular datasets, and to discover all systems where data is found that meets some user-defined criteria. This tool has been readily adopted by users at GE Capital and in a short time has already become a business-critical application, with over 2,200 data systems and over 1,000 data flows captured.

Parallel Factor Analysis (PARAFAC) is used in many scientific disciplines to decompose multimodal datasets ('tensors') into principal factors to uncover multilinear relationships in the data. Today's popular implementations of PARAFAC are single-server solutions that do not scale well to big datasets. This paper presents the design, implementation, and testing of a Big Data-enabled Parallel PARAFAC algorithm written in Java and run via Hadoop. To optimize performance across diverse datasets, three computational modes have been implemented: (i) dense, (ii) sparse, and (iii) hybrid. The input tensor is first divided into slices for parallel processing, then a one-pass test is performed on each slice to determine its size and sparsity, and finally, the best mode for each slice is chosen to minimize the overall runtime. For tensors with non-uniform data density distributions, we demonstrate that a per slice selection of the computational mode minimizes the runtime compared to using the same mode for all slices. In one representative example, the runtime with auto-selected mode was 3% and 25% faster than pure hybrid and sparse mode executions, respectively, resulting in a faster execution of PARAFAC on Big Data tensors than previously possible.

The era of precision medicine is best exemplified by the growing reliance on next-generation sequencing (NGS) technologies to provide improved disease diagnosis and targeted therapeutic selection. Well-established NGS data analysis software tools, in their unmodified form, can take days to identify and interpret single nucleotide and structural variations in DNA for a single patient. To improve sample analysis throughput, we developed a highly parallel end-to-end next-generation DNA sequencing data analysis pipeline in Hadoop. In our pipeline, each step is parallelized not only across samples but also within each individual sample, achieving a 30× speedup over a single server workflow execution. Furthermore, we extensively evaluate the viability of having our Hadoop-based pipeline as part of a larger commercial genomic services offering-we demonstrate how our pipeline scales sub-linearly both with the number of samples being analyzed and with the depth of coverage of those samples. In particular, on our commodity cluster, 10× as many samples resulted in only a 2.24× increase in the execution time, and a 4× increase in coverage depth resulted in only a 2.53× growth in execution time. We anticipate that such improvements will allow large cohort populations to be analyzed in parallel, and can fundamentally change the way DNA sequencing analyses are used by both researchers and clinicians.

With the advent of Big Data technologies, organizations can efficiently store and analyze more data than ever before. However, extracting maximal value from this data can be challenging for many reasons. For example, datasets are often not stored using human-understandable terms, making it difficult for a large set of users to benefit from them. Further, given that different types of data may be best stored using different technologies, datasets that are closely related may be stored separately with no explicit linkage. Finally, even within individual data stores, there are often inconsistencies in data representations, whether introduced over time or due to different data producers. These challenges are further compounded by frequent additions to the data, including new raw data as well as results produced by large-scale analytics. Thus, even within a single Big Data environment, it is often the case that multiple rich datasets exist without the means to access them in a unified and cohesive way, often leading to lost value. This paper describes the development of a Big Data management infrastructure with semantic technologies at its core to provide a unified data access layer and a consistent approach to analytic execution. Semantic technologies were used to create domain models describing mutually relevant datasets and the relationships between them, with a graphical user interface to transparently query across datasets using domain-model terms. This prototype system was built for GE Power & Water's Power Generation Products Engineering Division, which has produced over 50TB of gas turbine and component prototype test data to date. The system is expected to result in significant savings in productivity and expenditure.

With an exponential increase in time series sensor data generated by an ever-growing number of sensors on industrial equipment, new systems are required to efficiently store and analyze this “Industrial Big Data.” To actively monitor industrial equipment there is a need to process large streams of high velocity time series sensor data as it arrives, and then store that data for subsequent analysis. Historically, separate systems would meet these needs, with neither system having the ability to perform fast analytics incorporating both just-arrived and historical data. In-memory data grids are a promising technology that can support both near real-time analysis and mid-term storage of big datasets, bridging the gap between high velocity and high volume big time series sensor data. This paper describes the development of a prototype infrastructure with an in-memory data grid at its core to analyze high velocity (>100,000 points per second), high volume (TB's) time series data produced by a fleet of gas turbines monitored at GE Power & Water's Remote Monitoring & Diagnostics Center.

Traditional sales support systems offer an insular view of companies, providing information on a target company and that company alone. This view presents each company as if it operates independently from its surroundings. However, a company can be more effectively evaluated when it is viewed not in isolation, but in the context of its network of customers and suppliers. Here we describe a system developed for GE Capital that enables sales representatives to analyze a target company's network of business relationships to gain deeper insights into its status. Mining company networks enables novel approaches to generating sales leads by taking advantage of information about the health and status of the suppliers and customers to which the company is linked. The system allows sales reps to visualize company networks and provides a variety of filters and views to mine the network information. This gives the sales reps deeper insights into their prospect pool, enabling them to be more knowledgeable and generate more actionable sales leads.

Sales representatives must have access to meaningful and actionable intelligence about potential customers to be effective in their roles. Historically, GE Capital Americas sales reps identified leads by manually searching through news reports and financial statements either in print or online. Here we describe a system built to automate the collection and aggregation of information on companies, which is then mined to identify actionable sales leads. The Financing Lead Triggers system is comprised of three core components that perform information fusion, knowledge discovery and information visualization. Together these components extract raw data from disparate sources, fuse that data into information, and then automatically mine that information for actionable sales leads driven by a combination of expert-defined and statistically derived triggers. A web-based interface provides sales reps access to the company information and sales leads in a single location. The use of the Lead Triggers system has significantly improved the performance of the sales reps, providing them with actionable intelligence that has improved their productivity by 30-50%. In 2010, Lead Triggers provided leads on opportunities that represented over $44B in new deal commitments for GE Capital.

Quantum computing has the potential to revolutionize the field of computing, but with hardware and algorithms unlike any in use today. Due to the primitive state of existing quantum hardware, simulation is one of the most effective methods for studying quantum computing issues. Our team previously developed a general-purpose simulator capable of modeling arbitrary quantum algorithms executing on any hardware device. The simulator performs over a thousand matrix multiplications per step as part of its operations. To improve the simulator performance, optimizations were designed to dynamically restructure the problem. The resulting calculations were then distributed across a cluster. These enhancements reduced both the order of the simulator operations and the memory overhead, achieving an overall performance improvement of 99.94% from the initial implementation of a key quantum algorithm, reducing the simulator run time for this algorithm from two days on a single processor to under two minutes on a cluster.

A robust method for automating the tuning and maintenance of fuzzy decision-making systems is presented. A configurable multi-stage mutation-based evolutionary algorithm optimally tunes the decision thresholds and internal parameters of fuzzy rule-based and case-based systems that decide the risk categories of insurance applications. The tunable parameters have a critical impact on the coverage and accuracy of. decision-making, and a reliable method to optimally tune these parameters is critical to the quality of decision-making and maintainability of these systems.

Many automobile accidents could be avoided if the correct amount of brake pressure had been applied at the proper time. In this paper we propose the use of a fuzzy control system that could be embedded in an automobile to remove the driver's reaction time from the situation. The system, consisting of two fuzzy controllers connected in cascade, analyzes possible accident situations based on the vehicle separation distance, their relative velocity and a static friction coefficient that represents the road conditions. The controller determines the sufficient brake pressure required to prevent collisions while providing a smooth ride for the vehicle's passengers. This smooth performance was achieved by using genetic algorithms to tune the fuzzy controllers' parameters.