Formal methods are increasingly adopted in systems where reliability and correctness are critical, enabled by improvements in tool usability, speed, and automation. This industrial experience report presents three projects at Huawei Cloud showcasing different trade-offs in investment and assurance levels. We applied probabilistic concurrency testing, model checking, and deductive verification to two foundational services in the database and networking domains: the K2 transactional key-value store and the Global Server Load Balancer (GSLB).

Partial Differential Equations (PDEs) underpin many scientific phenomena, yet traditional computational approaches often struggle with complex, nonlinear systems and irregular geometries. This paper introduces the AMG method, a Multi-Graph neural operator approach designed for efficiently solving PDEs on Arbitrary geometries. AMG leverages advanced graph-based techniques and dynamic attention mechanisms within a novel GraphFormer architecture, enabling precise management of diverse spatial domains and complex data interdependencies. By constructing multi-scale graphs to handle variable feature frequencies and a physics graph to encapsulate inherent physical properties, AMG significantly outperforms previous methods, which are typically limited to uniform grids. We present a comprehensive evaluation of AMG across six benchmarks, demonstrating its consistent superiority over existing state-of-the-art models. Our findings highlight the transformative potential of tailored graph neural operators in surmounting the challenges faced by conventional PDE solvers. Our code and datasets are available on https://github.com/lizhihao2022/AMG.

The rise of global data-driven applications has made geo-distributed hybrid transactional and analytical processing (HTAP) databases increasingly desirable. Existing distributed HTAP systems provide users with good performance on both transactions and analytical queries, and this good performance is scalable across a large number of data nodes. Unfortunately, these systems either provide weak consistency or incur bad data freshness when deployed geographically. In this paper, we present P erseus , a scalable HTAP database that enforces strong consistency for both transactions and analytical queries. To handle consistency efficiently, P erseus augments the classical dependency graph in concurrency control protocols to explicitly record the versions of data and their complete dependencies, implying which data needs to be read together in a snapshot. To minimize data staleness on analytical queries (another important goal of HTAP), P erseus further introduces a new dynamic snapshot algorithm that chooses updates selectively. Extensive evaluation results show that, compared to the HTAP databases with even weaker consistency, P erseus achieves up to 90% lower visibility delay, a metric of data freshness, capturing the time interval during which transactional updates are committed to the database and can be visible to analytical queries. Besides, Perseus is scalable across many nodes and robust to network instability.

TrueTime clocks (TTCs) that offer accurate and reliable time within limited uncertainty bounds have been increasingly implemented in many clouds. Multi-region data stores that seek decentralized synchronization for high performance represent an ideal application of TTC. However, the co-designs between the two often failed to realize their full potential. This paper proposes K2, a multi-region data store that explores the opportunity of using TTC for distributed transactions. Compared to its pioneer, Google Spanner, K2 augments TTC's semantics in three core design pillars. First, K2 carries a new timestamp-generating scheme that is capable of providing a small time uncertainty bound at scale. Second, K2 revitalizes existing multi-version timestamp-ordered concurrency control to realize multi-version properties for read-write transactions. Third, K2 introduces a new TTC-based visibility control protocol that provides efficient reads at replicas. Our evaluation shows that, K2 achieves an order of magnitude higher transaction throughput relative to other geo-distributed transaction protocols while ensuring a lower visibility delay at asynchronous replicas.

Encrypted Databases (EDBs) are essential for protecting sensitive data outsourced to public clouds, enabling diverse index-based queries over encrypted data. However, existing EDB indexes often incur high storage overhead and performance degradation, primarily due to the poor compressibility of pseudorandom encrypted values, which leads to frequent accesses to slower persistent storage as indexes outgrow main memory. We introduceECStore, the first EDB that supports compressible and efficient indexing. Observing that EDB indexes are used solely for lookups and never decrypted, we designECTree, a cryptographic hash-based index structure in which each node is a compressible bit-string identifier that conceals plaintext keys.ECTreeenables logarithmic-time encrypted search via a novel membership testing mechanism. To address false positives arising in dynamic workloads, we introduceDirected View Check(DVC), which detects inaccuracies and avoids redundant traversals. Additionally,ECTree's Merkle-tree-like structure supports encrypted query authentication, resisting server compromise. Extensive evaluations show thatECStorecan achieve up to 94.7% lower latency and 10.5x higher throughput on popular benchmarks compared to notable EDBs.

Abstract To provide applications with the ability to analyze fresh data and eliminate the time-consuming ETL workflow, hybrid transactional and analytical (HTAP) systems have been developed to serve online transaction processing and online analytical processing workloads in a single system. In recent years, HTAP systems have attracted considerable interest from both academia and industry. Several new architectures and technologies have been proposed. This paper provides a comprehensive overview of these HTAP systems. We review recently published papers and technical reports in this field and broadly classify existing HTAP systems into two categories based on their data formats: monolithic and hybrid HTAP. We further classify hybrid HTAP into four sub-categories based on their storage architecture: row-oriented, column-oriented, separated, and hybrid. Based on such a taxonomy, we outline each stream’s design challenges and performance issues (e.g., the contradictory format demand for monolithic HTAP). We then discuss potential solutions and their trade-offs by reviewing noteworthy research findings. Finally, we summarize emerging HTAP applications, benchmarks, future trends, and open problems.

The advent of data-intensive applications has fueled the evolution of hybrid transactional and analytical processing (HTAP). To support mixed workloads, distributed HTAP databases typically maintain two data copies that are specially tailored for data freshness and performance isolation. In particular, a copy in a row-oriented format is well-suited for OLTP workloads, and a second copy in a column-oriented format is optimized for OLAP workloads. Such a hybrid design opens up a new design space for query optimization: plans can be optimized over different data formats and can be executed over isolated resources, which we term hybrid plans. In this paper, we demonstrate that hybrid plans can largely benefit query execution (e.g., up to 11x speedups in our evaluation). However, we also found these benefits will potentially be at the cost of sacrificing data freshness or performance isolation since traditional optimizers may not precisely model and schedule the execution of hybrid plans on real-time updated HTAP databases. Therefore, we propose Metis, an HTAP-aware optimizer. We show, both theoretically and experimentally, that using the proposed optimizations, a system can largely benefit from hybrid plans while preserving isolated performance for OLTP and OLAP, and these optimizations are robust to the changes in workloads.

Cloud-native databases have become the de-facto choice for mission-critical applications on the cloud due to the need for high availability, resource elasticity, and cost efficiency. Meanwhile, driven by the increasing connectivity between data generation and analysis, users prefer a single database to efficiently process both OLTP and OLAP workloads, which enhances data freshness and reduces the complexity of data synchronization and the overall business cost. In this paper, we summarize five crucial design goals for a cloud-native HTAP database based on our experience and customers' feedback, i.e., transparency, competitive OLAP performance, minimal perturbation on OLTP workloads, high data freshness, and excellent resource elasticity. As our solution to realize these goals, we present PolarDB-IMCI, a cloud-native HTAP database system designed and deployed at Alibaba Cloud. Our evaluation results show that PolarDB-IMCI is able to handle HTAP efficiently on both experimental and production workloads; notably, it speeds up analytical queries up to ×149 on TPC-H (100GB). PolarDB-IMCI introduces low visibility delay and little performance perturbation on OLTP workloads (<5%), and resource elasticity can be achieved by scaling out in tens of seconds.

A distributed database utilizing the wide-spread edge computing servers to provide low-latency data access with the serializability guarantee is highly desirable for emerging edge computing applications. In an edge database, nodes are divided into regions, and a transaction can be categorized as intra-region (IRT) or cross-region (CRT) based on whether it accesses data in different regions. In addition to serializability, we insist that a practical edge database should provide low tail latency for both IRTs and CRTs, and such low latency must be scalable to a large number of regions. Unfortunately, none of existing geo-replicated serializable databases or edge databases can meet such requirements.

Quantum computing technology has reached a second renaissance in the last decade. However, in the NISQ era pointed out by John Preskill in 2018, quantum noise and decoherence, which affect the accuracy and execution effect of quantum programs, cannot be ignored and corrected by the near future NISQ computers. In order to let users more easily write quantum programs, the compiler and runtime system should consider underlying quantum hardware features such as decoherence. To address the challenges posed by decoherence, in this paper, we propose and prototype QLifeReducer to minimize the qubit lifetime in the input OpenQASM program by delaying qubits into quantum superposition. QLifeReducer includes three core modules, i.e., the parser, parallelism analyzer and transformer. It introduces the layered bundle format to express the quantum program, where a set of parallelizable quantum operations is packaged into a bundle. We evaluate quantum programs before and after transformed by QLifeReducer on both real IBM Q 5 Tenerife and the self-developed simulator. The experimental results show that QLifeReducer reduces the error rate of a quantum program when executed on IBMQ 5 Tenerife by 11%; and can reduce the longest qubit lifetime as well as average qubit lifetime by more than 20% on most quantum workloads.