Data processing systems face a challenge to support increasingly diverse workloads efficiently. At the same time, they are already bloated with internal complexity and it is not clear how new hardware can be supported sustainably.

We propose a layered query compilation framework with open intermediate representations that applies traditional query optimizations as compiler passes. This enables us to support cross-domain optimizations. Furthermore, we introduce a unified abstraction layer based on declarative sub-operators, as a step towards a future-proof execution of diverse workloads on modern hardware. We demonstrate the benefits by implementing the ideas into our compiling query engine, LingoDB. We show that our system can support a variety of workloads and complex operations with relatively low implementation effort, while also providing competitive performance to state-of-the-art systems.

 

Problem

The exponential expansion of the global datasphere, driven by use cases such as click-streaming, IoT, and edge computing, as well as the increasing range of data-intensive workloads beyond pure SQL workloads, requires flexible data systems that can adapt to a wide range of workloads.

In the current era, a Moore-like increase in transistors in integrated circuits cannot be maintained. Performance improvements require efficient use of multiple cores and specialized hardware like CPUs, GPUs, TPUs, ASICs, and other ICs. In addition, through virtualization in today's data system deployments, it is typical that large amounts of the allocated resources, which play a significant part in the overall system cost, remain unused.

The cause of this is the widespread design decisions of execution engines, where the workloads are treated as black-box integrations, meaning that they are not transparent to the adapting system. Consequently, workload optimization cannot be performed, and data formats must be serialized. In addition, the underlying hardware gets white-box integrated, meaning that developers have to perform manual hardware integration for a chosen hardware target.

A significant improvement would be to work with a hardware-agnostic data processing system that performs these optimizations automatically (meaning the underlying hardware is black-box) and allows for white-box integrations of workloads.

4. Goal

The overall goal of this approach is to achieve what the speaker refers to as “Future-proof Data Systems”. This defines a system that is “adjustable and extensible for future workloads” as well as being “decoupled from the ongoing hardware evolution”. The latter refers to the segregation of specialized hardware mentioned in 3.

5. Approach

Today's execution engines consist of complex monolithic structures like query optimization or query execution. The general approach presented in the lecture is based on the idea of reversing the aforementioned way execution engines integrate user-defined workloads and the underlying hardware. This requires breaking up these monolithic structures.

The crucial part of this approach is the avoidance of performance tradeoffs when trying to achieve this flexible and extensible abstraction layer. When consolidating the white-box and black-box design described in 3, the base layer/monolith would be reworked into a system of open intermediate representations (IRs) instead of a classical closed IR of the user-supplied workload. The benefit of this approach is that subsequent IRs of different domains can be combined without any overhead to the user. A second layer would be responsible for query optimization in the form of compiler passes. This is necessary to make cross-domain optimization feasible. Lastly, the layered query execution layer enables reasoning about the type of performed optimizations of underlying abstraction layers until an executable is reached.

6. LingoDB

LingoDB is a research prototype of a future-proof data processing system that leverages advanced compiler technology. Carefully crafted additional abstraction layers increase flexibility and extensibility. LingoDB maintains competitive performance compared to other data processing frameworks such as DuckDB [9], HyPer [5], and Umbra [8].

7. Multi-Level Intermediate Representation (MLIR)

MLIR is a compiler infrastructure project similar to LLVM (Low Level Virtual Machine). While LLVM represents the basis of many modern programming languages such as C, C++, Rust, Swift, or Julia, MLIR is a more recent iteration of these frameworks aiming to address some limitations of the LLVM IR, such as making the addoption machine learning, domain-specific languages, and heterogeneous hardware targets feasible [6]. MLIR uses a unified IR that can be mapped onto different domain-specific IRs, such as low-level assembly-like languages like LLVM IR, but also high-level representations such as TensorFlow graphs [3]. MLIR is used within LingoDB for this specific purpose. It combines multiple IRs within the system and this results in LingoDB be able to support a wide range of workloads.

8. LingoDB for Future-proof Data Processing

Fig.1 - Overview of LingoDB's framework.

LingoDB parses different types of queries. For each query, it creates an MLIR module with custom IR dialects. The module optimizes and compiles the query with compiler passes and custom lowerings, yielding an LLVM IR for compilation. Its runtime library is based on Apache Arrow and translates the LLVM IR into LLVM and machine code (Fig.1).

The following section discusses the optimization/compilation framework of LingoDB in detail.

8.1 Optimization and Compilation Framework

8.1.1. Custom IR Dialects

The approach presented in the lecture defines four custom IR dialects within MLIR:

• relalg: Relational operators on tuple streams.
• db: Types and scalar operations used by SQL.
• dsa: Data structures and algorithms.
• util: Low-level utility types and operations.

The combination of the dialects mentioned above can represent everything happening in a database system in the context of MLIR [2].

An important goal is to implement relational operations in a non-monolithic way. As the implementation is complex, the effective use of underlying hardware is nontrivial, and more complex operators get added to the SQL standard; this is a difficult challenge. Additionally, the rising popularity of user-defined functions (UDFs) requires more than just considering relational algebra.

8.1.2. Sub-Operators

An additional abstraction layer is introduced due to the observations above. Sub-operators are simple code blocks that can be reasoned about and implemented efficiently [2].

Conceptually, sub-operators work on tuple streams, allowing users to compose and represent existing SQL operators. LingoDB emphasizes explicit declarative state management to decouple sub-operators and states. Additionally, references to the states allow state navigation for complex transformations, while explicit control flows ensure that sub-operators remain simple and reusable. Views, as properties atop states, further improve the system's flexibility without needing state replication.

Besides representing relational operators, they expose an interface for user-defined operators and algorithms so the users can specify their operators and algorithms with sub-operators. They also enable automatic optimization for modern hardware.

Any physical plan can be expressed as a sub-operator plan. The implemented, optimized sub-operator plan will then be translated into generated code (Fig.2).

Fig.2 - Translation of a physical plan into a plan containing sub-operators.

8.1.3. Compiler Passes

Compiler passes can represent all the standard optimizations in modern query optimizers using the custom IR dialects. Also, it is possible to benefit from existing passes in MLIR [2]. Interleaved compiler passes introduce more flexibility compared to the monolithic compilation of IRs.

The simplicity of sub-operators and having explicit states as well as explicit access to the states enables powerful optimizations applied as interleaved compiler passes.

Auto-parallelization can now be applied as a single compiler pass: Sub-operator states can be used to detect read-write, write-read, and write-write conflicts. As a reaction, the compiler pass applies techniques for avoiding detected conflicts as code transformations.

8.2. Memory-centric Systems

Under the software layer, the underlying hardware has become very rich in different memory and compute components. Optimizing programs in software has become challenging due to the complex interplay of these different types of memory with computing parts of the machines. The speaker proposes to create another abstraction layer around a memory-centric system to achieve the independence of program models from the changing hardware over time.

8.2.1. Memory Regions (MR)

The abstraction layer should serve logical memory regions that get a set of declarative properties attached to them. Deploying the logical memory region on a physical memory part should be based on the attached properties. Afterward, the connected compute parts can perform the requested tasks.

In the first step, the new abstraction layer exposes memory regions. Then, the mapping of regions happens at runtime. Memory types can be chosen depending on whether the task runs on the CPU or GPU. The memory is then selected based on the task's declarative properties, data dependencies, task placement, and the availability of the physical resources (Fig.3).

Fig.3 - Abstraction layer exposing different memory regions.

8.2.2. Typed Memory Regions

Each memory region can also be typed based on its use case. The speaker proposes global state, global scratch, and private scratch. The global state is used for coherent and synchronous communication. Exchanging data can be done asynchronously and coherently and would be typed as a global scratch. Local computations should be typed as private scatch.

Different dataflow systems can map their operations into these typed memory regions based on their purpose and properties. DBMS operations for synchronization would be typed as global state, while the input data for ML and AI can typed as global scatch.

8.2.3. Control Plane Tasks

Switching to the control plane, Prof. Dr. Jana Giceva mentions basic memory region ownership schemes, like singular, transferred, or shared ownership. Managing the ownership is essential to avoid data movement or copying. Secondly, the control planes manage the lifetimes of the memory regions. Therefore, garbage collection needs a lightweight metadata system to keep track of the usage of memory regions. Further tasks like resource monitoring and deployment are assigned to the control plane.

9. Conclusion

In the presentation by Dr. Prof. Jana Giceva, the main goal of making future-proof data processing is achievable by an open compilation and optimization framework. The proposed ideas add abstraction layers; however, the approach avoids a trade between higher abstractions and performance.

The central concept relies on whiteboxing the optimization and compilation framework by making them more interchangeable in each step by adding declarative suboperators and intermediate formats. The research prototype LingoDB implements the ideas presented. Its comparison with DuckDB, Hyper, and Umbra shows that a hardware-agnostic data processing system with carefully crafted abstraction layers still achieves competitive performance while increasing flexibility and extensibility.

The last topic of her presentation covered a request for blackboxing the underlying hardware to be independent of the rapid evolution of these; she introduced a memory-centric system. Her proposal offers an abstraction for selecting the correct memory type based on declarative attributes.

10. References

[1]

Jana Giceva, Gustavo Alonso, Timothy Roscoe, and Tim Harris. 2014. Deployment of query plans on multicores. Proceedings of the VLDB Endowment 8, 3 (November 2014), 233–244. https://doi.org/10.14778/2735508.2735513

[2]

Michael Jungmair and Jana Giceva. 2023. Declarative Sub-Operators for Universal Data Processing. Proceedings of the VLDB Endowment 16, 11 (July 2023), 3461–3474. https://doi.org/10.14778/3611479.3611539

[3]

Michael Jungmair, André Kohn, and Jana Giceva. 2022. Designing an open framework for query optimization and compilation. Proceedings of the VLDB Endowment 15, 11 (July 2022), 2389–2401. https://doi.org/10.14778/3551793.3551801

[4]

Kaan Kara, Jana Giceva, and Gustavo Alonso. 2017. FPGA-based Data Partitioning. In Proceedings of the 2017 ACM International Conference on Management of Data (SIGMOD '17), May 2017. Association for Computing Machinery, New York, NY, USA, 433–445. https://doi.org/10.1145/3035918.3035946

[5]

Alfons Kemper and Thomas Neumann. 2011. HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots. In 2011 IEEE 27th International Conference on Data Engineering, April 2011. 195–206. https://doi.org/10.1109/ICDE.2011.5767867

[6]

Chris Lattner, Mehdi Amini, Uday Bondhugula, Albert Cohen, Andy Davis, Jacques Pienaar, River Riddle, Tatiana Shpeisman, Nicolas Vasilache, and Oleksandr Zinenko. 2021. MLIR: Scaling Compiler Infrastructure for Domain Specific Computation. In 2021 IEEE/ACM International Symposium on Code Generation and Optimization (CGO), February 2021. 2–14. https://doi.org/10.1109/CGO51591.2021.9370308

[7]

Darko Makreshanski, Jana Giceva, Claude Barthels, and Gustavo Alonso. 2017. BatchDB: Efficient Isolated Execution of Hybrid OLTP+OLAP Workloads for Interactive Applications. In Proceedings of the 2017 ACM International Conference on Management of Data (SIGMOD '17), May 2017. Association for Computing Machinery, New York, NY, USA, 37–50. https://doi.org/10.1145/3035918.3035959

[8]

Thomas Neumann and Michael Freitag. Umbra: A Disk-Based System with In-Memory Performance.

[9]

Mark Raasveldt and Hannes Mühleisen. 2019. DuckDB: An Embeddable Analytical Database. In Proceedings of the 2019 International Conference on Management of Data (SIGMOD '19), June 2019. Association for Computing Machinery, New York, NY, USA, 1981–1984. https://doi.org/10.1145/3299869.3320212