Group-By-Aggregate (GBA) queries are fundamental in data analysis, allowing users to organize data by specific attributes and apply aggregate functions, such as sum, average, and count, to generate insights from large datasets. Traditionally, DBMSs handle GBA queries through either sort-based or hashbased methods, each with unique strengths and limitations. Sortbased approaches, while effective in certain contexts, can become computationally expensive when few groups are involved due to extensive record comparisons. On the other hand, hash-based methods are efficient with larger group counts but are memoryintensive and may suffer from hash collisions, depending on data distribution and hashing functions.This paper presents an empirical performance study focused on traditional sort- and hash-based GBA operators, introducing a robust hybrid approach that unifies their strengths to deliver consistent, high performance across diverse data conditions. The hybrid GBA method combines hash-based grouping and early aggregation for the local phase with sort-based aggregation for the global phase. It forwards partially aggregated results when memory limits are reached, reducing data volume and minimizing memory usage and disk I/O. By eliminating the need to choose between sort and hash GBA operators and removing dependency on precise statistics or query optimizer intervention, the hybrid method significantly enhances query execution reliability. Extensive testing on Wisconsin, TPC-H, and TPC-DS benchmark datasets demonstrates that the hybrid approach consistently matches the performance of the bestsuited algorithm—whether sort or hash—in each scenario, across varying dataset sizes and complexities. This unified solution replaces separate GBA algorithms, providing a flexible, easily adaptable, and highly efficient method for processing GBA queries in big data management systems.

Over the years, data management systems have undergone significant transformations adapting to advancements in hardware, diverse data types, and varying workloads. Numerous architectures have been proposed to more effectively address the customer needs, taking into account rapidly evolving factors like emerging data types, shifting workload and access patterns, and modern hardware innovations. With emergence of cloud infrastructures and their improved capability in seamless scaling, the shared-nothing architecture, which dominated the field for about three decades, is now being replaced with disaggregated architectures. The considerable cost of cloud resources has significantly increased the interest in further disaggregating resources, enabling them to scale independently of one another as required. Consequently, cost-sharing via multi-tenancy and the benefits of pay-for-use billing policies in serverless computing have become pivotal in shaping the design of new database architectures. This paper provides a concise overview of database architecture evolution and their triggers for change, the contemporary state-of-the-art structures, and highlights open research problems for future exploration.

In database management systems (DBMSs) that handle multiple concurrent queries, adapting to fluctuating workloads is crucial. This flexibility allows the DBMS to revise decisions based on current workload and available resources. As memory availability changes with the arrival or completion of queries, having memory-intensive operators like the Hybrid Hash Join that dynamically adapt is vital. This paper introduces a new memory-adaptive Hash-Based join algorithm design implemented in Apache AsterixDB and evaluates its responsiveness to memory variability.

Hybrid Hash Join (HHJ) has proven to be one of the most efficient and widely-used join algorithms. While HHJ's performance depends largely on accurate statistics and information about the input relations, it may not always be practical or possible for a system to have such information available. HHJ's design depends on many details to perform well. This paper is an experimental and analytical study of the trade-offs in designing a robust and dynamic HHJ operator. We revisit the design and optimization techniques suggested by previous studies through extensive experiments, comparing them with other algorithms designed by us or used in related studies. We explore the impact of the number of partitions on HHJ's performance and propose a new lower bound for the number of partitions. We design and evaluate different partition insertion techniques to maximize memory utilization with the least CPU cost. Additionally, we consider a comprehensive set of algorithms for dynamically selecting a partition to spill and compare the results against previously published studies. We then present and evaluate two alternative growth policies for spilled partitions. These algorithms have been implemented in the context of Apache AsterixDB and evaluated under different scenarios such as variable record sizes, different distributions of join attributes, and different storage types, including HDD, SSD, and Amazon Elastic Block Store (Amazon EBS).

Benchmarks have always been one of the greatest assets in evaluating a Data Management System (DBMS) performance and providing a standard way to compare different DBMSs from various angles. As DBMSs evolve over time, new benchmarks are created, and the older ones need to advance and adapt in order to continue to be a valid and capable comparison tool. Many of the standard benchmarks use synthetic data and as such may not be able to capture the complexity of real data and its relationships correctly. The Wisconsin Benchmark was one of the first and main benchmarking tools made four decades ago by Dewitt et al at the University of Wisconsin. One of the most powerful features of this benchmark is that its relations are designed so their structure and distribution of attributes is easy to understand and control. While the Wisconsin Benchmark was a very powerful and widely-used benchmark years ago, it is given less attention in current studies. We believe that the Wisconsin Benchmark and its carefully designed relations can be utilized and provide capabilities that are unique and useful.. In this paper, we present a flexible, easy-to-use and scalable JSON Data Generator implemented in java based on the Wisconsin Benchmark Data Generator description, with more advanced features to provide relations and attributes closer to real-world data. Attribute skewness and variable length records using different size distributions are some of these newly added features. It is a ready-to-use, parameterized, and scalable data generator tool that since its development has been used in several AsterixDB's [4] performance benchmarking and publications [13,14,17,18], and we believe that can be useful to many others. The source code and more information are provided at [1].

Re-evaluating the Performance Trade-offs for Hash-Based Multi-Join Queries Share on Author: Shiva Jahangiri University of California, Irvine, Irvine, CA, USA University of California, Irvine, Irvine, CA, USASearch about this author Authors Info & Claims SIGMOD '20: Proceedings of the 2020 ACM SIGMOD International Conference on Management of DataJune 2020 Pages 2845–2847https://doi.org/10.1145/3318464.3384406Online:11 June 2020Publication History 1citation131DownloadsMetricsTotal Citations1Total Downloads131Last 12 Months44Last 6 weeks4 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access