Data Analytics is at the core of almost all modern ap-plications ranging from science and finance to healthcare and web applications. The evolution of data analytics over the last decade has been dramatic - new methods, new tools and new platforms - with no slowdown in sight. This rapid evolution has pushed the boundaries of data analytics along several axis including scalability especially with the rise of distributed infrastructures and the Big Data era, and interoperability with diverse data management systems such as relational databases, Hadoop and Spark. However, many analytic application developers struggle with the challenge of production deployment. Recent experience suggests that it is difficult to deliver modern data analytics with the level of reliability, security and manageability that has been a feature of traditional SQL DBMSs. In this tutorial, we discuss the advances and innovations introduced at both the infrastructure and algorithmic levels, directed at making analytic workloads scale, while paying close attention to the kind of quality of service guarantees different technology provide. We start with an overview of the classical centralized analytical techniques, describing the shift towards distributed analytics over non-SQL infrastructures. We contrast such approaches with systems that integrate analytic functionality inside, above or adjacent to SQL engines. We also explore how Cloud platforms' virtualization capabilities make it easier - and cheaper - for end users to apply these new analytic techniques to their data. Finally, we conclude with the learned lessons and a vision for the near future.

Teradata Vantage is a platform for integrating a broad range of analytical functions and capabilities with the Teradata's SQL engine. One of the main challenges in optimizing the execution of these analytical functions is that many of them are not only black boxes, but also have polymorphic nature, i.e., their behavior and properties may change depending on the invocation context. In this paper, we first demonstrate the inherent complexity in optimizing polymorphic functions, and then present the Vantage's Collaborative Optimizer , which is a cross-platform optimizer designed for optimizing the analytical functions invoked from within the SQL engine. The Collaborative Optimizer is the industry-first effort towards enabling analytics-aware optimizations over polymorphic analytical functions. We present a novel markup language-based approach for expressing the functions' polymorphic properties via a set of well-defined instructions. The Collaborative Optimizer uses these instructions at query time to infer the corresponding properties, and then decide on the applicable optimizations. From several possible optimizations, we showcase two core optimizations, namely "projection push" and "predicate push" , which aim at optimizing the data movement to and from the analytical functions. The experiments using the Teradata-MLE analytical system demonstrate the expressiveness power and flexibility of the proposed markup language. Moreover, benchmark and real-world customer queries show the significant performance gain that the Collaborative Optimizer brings to the Vantage system.

Big data characterized by variety can be divided into 3 principal categories: numeric structured data, semi-structured data, and unstructured multimedia data involving audio, video, and text. Decision making requires multiple analytical engines suitable for each type of data, programming languages, algorithms, visualization tools, and user interfaces. More often than not, industrial analytics is conducted ad hoc by lashing together analytics components such as distributed data sources, analytics engines, and algorithms. This kind of piecemeal approach ignores scale, security, governance, reliability, model management and fault tolerance that are paramount for industrial strength analytics. A unified, versatile, and robust architecture that combines various components in a single integrated platform is the need of the hour. Teradata Vantage (TD Vantage) is such a platform for delivering production quality enterprise analytics at scale. In this paper, we outline the proposed TD Vantage (available in the market and under continuous development) that unifies data, engines, and algorithms operating in a seamless symphony. We will demonstrate its capabilities through three proofs of concept biz: image data using TensorFlow, text data using Spark, and transaction data using Aster (now renamed Machine Learning Engine or MLE), with Teradata orchestrating interactions among the various components.

Ease-of-use analytics at scale is the holy grail of industrial strength machine learning. In order to reap benefits from the mother-lode of business related data; tools, technologies, and analytical functions should operate in perpetual symphony to derive insightful business outcomes. While there have been advances in APIs, algorithms, and user interfaces, building an end to end workflow spanning data ingestion, data preparation, model training, model scoring, visualization and finally continuous improvement and model management received limited investment. In this paper we demonstrate an analytical workflow that integrates multiple analytical tools and techniques for image recognition wrapped in the model management framework. As analytics in industry is maturing, analytics implementations are no longer one-off, but are components of Analytics Operations known as AnalyticsOps. We introduce the notion of Model Quality Index (MQI) to track model performance. The MQI is similar to Process Capability Index (PCI) common in 6σ programs in manufacturing. Our solution combines relational databases (Teradata DB), Machine Learning (Teradata/Aster), Deep Learning (TensorFlow), Hadoop Distributed File System (HDFS), and user interface tools over a communication fabric (Teradata QueryGrid). In particular, we demonstrate a hand written word recognition use-case for an enterprise customer cast in a model management workflow for repeatable deployments across a range of businesses.

Ease-of-use analytics at scale is the holy grail of industrial strength machine learning. While there have been advances in APIs, algorithms, and user interfaces, building an end to end work flow involving data ingestion, data preparation, model training, model scoring, and visualization received limited investment and effort producing only marginal innovation. This paper outlines a proof of concept that demonstrates an analytical workflow that integrates multiple analytical tools and techniques for image recognition. The solution combines Relational Databases and Machine Learning (teradata), Deep Learning (TensorFlow), Distributed File System (HDFS), Graphical Processing Units, and user interface tools over a communication fabric (teradata QueryGrid). In particular, we demonstrate hand written word recognition through an application of Convolutional Neural Networks in TensorFlow and teradata's custom analytical functions to recognize first names and last names in the payee field section of financial negotiable instruments such as in a cheque. We hope that this paper will serve as a guide to successful implementation of analytical workflows in production.