Many organizations have embraced the "Lakehouse" data management paradigm, which involves constructing structured data warehouses on top of open, unstructured data lakes. This approach stands in stark contrast to traditional, closed, relational databases and introduces challenges for performance and stability of distributed query processors. Firstly, in large-scale, open Lakehouses with uncurated data, high ingestion rates, external tables, or deeply nested schemas, it is often costly or wasteful to maintain perfect and up-to-date table and column statistics. Secondly, inherently imperfect cardinality estimates with conjunctive predicates, joins and user-defined functions can lead to bad query plans. Thirdly, for the sheer magnitude of data involved, strictly relying on static query plan decisions can result in performance and stability issues such as excessive data movement, substantial disk spillage, or high memory pressure. To address these challenges, this paper presents our design, implementation, evaluation and practice of the Adaptive Query Execution (AQE) framework, which exploits natural execution pipeline breakers in query plans to collect accurate statistics and re-optimize them at runtime for both performance and robustness. In the TPC-DS benchmark, the technique demonstrates up to 25× per query speedup. At Databricks, AQE has been successfully deployed in production for multiple years. It powers billions of queries and ETL jobs to process exabytes of data per day, through key enterprise products such as Databricks Runtime, Databricks SQL, and Delta Live Tables.

In this paper we introduce a new method of performing direct solution of the harmonic balance Jacobian. For examples with moderate number of harmonics and moderate to strong nonlinearities, we demonstrate that the direct solver has far superior performance with a moderate increase in memory compared to the best preconditioned iterative solvers. This solver is especially suited for Fourier envelope analysis where the number of harmonics is small, circuits are nonlinear and Jacobian bypass can be used for additional speed. For examples with large number of harmonics and moderate to strong nonlinearities, the performance advantage is maintained but the memory requirements increase. We propose efficient preconditioners based on direct solution of harmonic balance matrices which provide the user with a memory-speed trade-off.

Typical analog and radio frequency (RF) circuit sizing optimization problems are computationally hard and require the handling of several conflicting cost criteria. Many researchers have used sequential stochastic refinement methods to solve them, where the different cost criteria can either be combined into a single-objective function to find a unique solution, or they can be handled by multiobjective optimization methods to produce tradeoff solutions on the Pareto front. This paper presents a method for solving the problem by the former approach. We propose a systematic method for incorporating the tradeoff wisdom inspired by the circuit domain knowledge in the formulation of the composite cost function. Key issues have been identified and the problem has been divided into two parts: a) normalization of objective functions and b) assignment of weights to objectives in the cost function. A nonlinear, parameterized normalization strategy has been proposed and has been shown to be better than traditional linear normalization functions. Further, the designers' problem specific knowledge is assembled in the form of a partially ordered set, which is used to construct a hierarchical cost graph for the problem. The scalar cost function is calculated based on this graph. Adaptive mechanisms have been introduced to dynamically change the structure of the graph to improve the chances of reaching the near-optimal solution. A correlated double sampling offset-compensated switched capacitor analog integrator circuit and an RF low-noise amplifier in an industry-standard 0.18mum CMOS technology have been chosen for experimental study. Optimization results have been shown for both the traditional and the proposed methods. The results show significant improvement in both the chosen design problems

This paper presents a multi-objective evolutionary algorithm based approach to yield-inclusive design space exploration of analog circuits. The yield objective function is included from the beginning in the optimization cycle, and is estimated economically by support vector regression models built from training data provided by the optimization process itself. We propose techniques to effectively handle the inaccurate nature of model-based yield estimates through various stages of the optimization process, and a two-level hybrid algorithm to keep the modeling cost reasonable. The experimental results presented for two real-world circuit examples look encouraging and show the effectiveness of the proposed method in providing designers with multiple trade-off choices

The knowledge of optimal design space boundaries of component circuits can be extremely useful in making good subsystem-level design decisions which are aware of the parasitics and other second-order circuit-level details. However, direct application of popular multi-objective genetic optimization algorithms were found to produce Pareto fronts with poor diversity for analog circuit problems. The paper proposes a novel approach to control the diversity of solutions by partitioning the solution space, using local competition to promote diversity and global competition for convergence, and by controlling the proportion of these two mechanisms by a simulated annealing based formulation. The algorithm was applied to extract numerical results on analog switched capacitor integrator circuits with a wide range of tight specifications. The results are found to be significantly better than traditional GA based uncontrolled optimization methods.