The cloud-native version of SAP IQ aims to reduce the storage and compute costs by storing data directly on object stores while benefiting from the greater elasticity and scale-out properties that are offered. In the cloud-native version of SAP IQ, the buffer manager has gone through a significant re-design for two major reasons. First, RAM is more expensive on the cloud; therefore, a buffer manager that relies exclusively on RAM would not have been a viable solution. Second, on object stores, read and write operations are generally associated with higher latencies. To counteract the impact of higher latencies without utilizing more RAM, we have extended SAP IQ's buffer manager with a second layer of cache, namely, the Extended Cache Manager (ECM). The ECM relies on fast solid state drives (SSDs). In this paper, we describe our experience with the design and implementation of the ECM, in particular, the design choices we had to make to overcome the fact that, when compared to object stores, SSDs have a much more limited I/O bandwidth.

As businesses continue to consume and produce ever-growing volumes of data, exploratory data analysis (EDA) is becoming an integral part of everyday operations. While online analytical processing (OLAP) systems in general - and column-oriented relational database management systems (RDBMS) in particular - are equipped with powerful tools to plough through petabytes of data, analytical queries may take seconds to execute, which is not always desirable in exploratory data analysis. Data scientists often need tools for fast visualization of data, and they are interested in identifying subsets of data that need further drilling-down before running computationally expensive analytical functions. In this paper, we describe our early work on extending SAP IQ (a disk-based columnar RDBMS) to support approximate query processing for exploratory data analysis using a technique known as query-time sampling. Specifically, we introduce two classes of novel samplers: (i) a stratified sampler with randomized row access to address the early-row bias problem in sampling, and (ii) hash-based equi-join samplers that are outlier-aware. We demonstrate how SAP IQ's polymorphic table function (PTF) technology can be utilized to implement these samplers as new query plan operators.

In this paper, we describe our journey of transforming SAP IQ into a relational database management system (RDBMS) that utilizes cheap, elastically scalable object stores on the cloud. SAP IQ is a three-decade old, disk-based, columnar RDBMS that is optimized for complex online analytical processing (OLAP) workloads. Traditionally, SAP IQ has been designed to operate on shared storage devices with strong consistency guarantees (e.g., high-caliber storage area network devices). Therefore, deploying SAP IQ on the cloud, as is, would have meant utilizing storage solutions such as NetApp or AWS EFS that provide a POSIX compliant file interface and strong consistency guarantees, but at a much higher monetary cost. These costs can accumulate easily to diminish the economies of scale that one would expect on the cloud, which can be undesirable. Instead, we have enhanced the design of SAP IQ to operate on cloud object stores such as AWS S3 and Azure Blob Storage. Object stores rely on a weaker consistency model, and potentially have higher latency; however, because of these design trade-offs, they are able to offer (i) better pricing, (ii) enhanced durability, (iii) improved elasticity, and (iv) higher throughput. By enhancing SAP IQ to operate under these design trade-offs, we have unlocked many of the opportunities offered by object stores. More specifically, we have extended SAP IQ's buffer manager and transaction manager, and have introduced a new caching layer that utilizes instance storage on AWS EC2. Experiments using the TPC-H benchmark demonstrate that we can gain an order of magnitude reduction in data-at rest storage costs while improving query and load performance.

Recent advances in Linked Data Management and the Semantic Web have led to a rapid increase in both the quantity as well as the variety of Web applications that rely on the SPARQL interface to query RDF data. Thus, RDF data management systems are increasingly exposed to workloads that are far more diverse and dynamic than what these systems were designed to handle. The problem is that existing systems rely on a workload-oblivious physical representation that has a fixed schema, which is not suitable for diverse and dynamic workloads. To address these issues, we propose a physical representation that is schemaless. The resulting flexibility enables an RDF dataset to be clustered based purely on the workload, which is key to achieving good performance through optimized I/O and cache utilization. Consequently, given a workload, we develop techniques to compute a good clustering of the database. We also design a new query evaluation model, namely, schemaless-evaluation that leverages this workload-aware clustering of the database whereby, with high probability, each tuple in the result set of a query is expected to be contained in at most one cluster. Our query evaluation model exploits this property to achieve better performance while ensuring fast generation of query plans without being hindered by the lack of a fixed physical schema.

The Resource Description Framework (RDF) is a standard for conceptually describing data on the Web, and SPARQL is the query language for RDF. As RDF is becoming widely utilized, RDF data management systems are being exposed to more diverse and dynamic workloads. Existing systems are workload-oblivious, and are therefore unable to provide consistently good performance. We propose a vision for a workload-aware and adaptive system. To realize this vision, we re-evaluate relevant existing physical design criteria for RDF and address the resulting set of new challenges.

Query plan caching eliminates the need for repeated query optimization, hence, it has strong practical implications for relational database management systems (RDBMSs). Unfortunately, existing approaches consider only the query plan generated at the expected values of parameters that characterize the query, data and the current state of the system, while these parameters may take different values during the lifetime of a cached plan. A better alternative is to harvest the optimizer's plan choice for different parameter values, populate the cache with promising query plans, and select a cached plan based upon current parameter values. To address this challenge, we propose a parametric plan caching (PPC) framework that uses an online plan space clustering algorithm. The clustering algorithm is density-based, and it exploits locality-sensitive hashing as a pre-processing step so that clusters in the plan spaces can be efficiently stored in database histograms and queried in constant time. We experimentally validate that our approach is precise, efficient in space-and-time and adaptive, requiring no eager exploration of the plan spaces of the optimizer.

Health Level Seven (HL7) is a prominent messaging standard in the eHealth domain, and with HL7 v2, it addresses only the messaging layer. However, HL7 implementations also deal with the other layers of interoperability, namely the business process layer and the communication layer. This need is addressed in HL7 v3 by providing a number of normative transport specification profiles. Furthermore, there are storyboards describing HL7 v3 message choreographies between specific roles in specific events. Having alternative transport protocols and descriptive message choreographies introduces great flexibility in implementing HL7 standards, yet, this brings in the need for test frameworks that can accommodate different protocols and permit the dynamic definition of test scenarios. In this paper, we describe a complete test execution framework for HL7-based systems that provides high-level constructs allowing dynamic set up of test scenarios involving all the layers in the interoperability stack. The computer-interpretable test description language developed offers a configurable system with pluggable adaptors. The Web-based GUIs make it possible to test systems over the Web anytime, anywhere, and with any party willing to do so.