Abstract
HYRISE is a main-memory database system designed to optimize mixed transactional (OLTP) and analytical (OLAP) workloads. By automatically partitioning tables into hybrid vertical layouts—combining row-like and column-like storage—HYRISE avoids the performance trade-offs inherent in traditional storage engines. Utilizing a sophisticated cost model based on CPU cache misses and a scalable graph-based partitioning algorithm, HYRISE achieves performance improvements ranging from 20% to 400% over pure row or column stores.
Introduction
Traditionally, the database market has been divided into row-based systems for Online Transaction Processing (OLTP) and columnar systems for Online Analytical Processing (OLAP). However, modern enterprise computing increasingly requires "real-time analytics," where complex analytical queries are executed directly on live transactional data. Running such mixed workloads on a single system typically introduces severe performance bottlenecks because the underlying physical storage is optimized for only one access pattern. HYRISE addresses this by designing a storage engine that creates bespoke hybrid layouts tailored to actual query access patterns.
System Architecture
To bridge the gap between OLTP and OLAP, HYRISE introduces a fine-grained hybrid storage model. Instead of storing a table strictly by rows or columns, it breaks tables into disjoint vertical partitions of varying widths, referred to as "containers." The system's core component is the automated Layouter, which determines the optimal widths for these containers.
As shown in the architecture, the Layout Manager uses workload data to drive the partitioning process, while the In-Memory Storage Manager handles the physical data containers. This separation allows the system to dynamically adapt the physical design as query patterns evolve.
Hybrid Layout Generation
In main-memory databases, the primary performance bottleneck is no longer disk I/O but CPU stalls caused by cache misses. HYRISE utilizes a Cache-Miss Cost Model that calculates expected L1 and L2 cache misses, data alignment, and prefetching policies for a given workload.
For tables with many attributes, the number of possible layouts grows exponentially. To handle this, HYRISE employs a Scalable Approximate Partitioning approach based on graph theory. Attributes are modeled as nodes, and their co-access frequencies are represented as edges.
By applying a multilevel k-way graph partitioner to find "min-cut subgraphs," HYRISE efficiently groups frequently co-accessed attributes together. This ensures that when the CPU fetches a cache line, it contains as much relevant data as possible, minimizing irrelevant data loading and reducing CPU stalls.
Evaluation and Results
The authors evaluated HYRISE using a benchmark derived from a real-world SAP enterprise resource planning (ERP) application. Unlike standard benchmarks, this dataset includes massive tables with over 200 attributes, reflecting the complexity of actual enterprise data. The workload consisted of a mix of 13 queries, including lookups, inserts, aggregations, and drill-downs.
Key results include:
- Performance Gains: HYRISE's hybrid layouts achieved a 20% to 400% speedup compared to traditional pure-column or pure-row designs.
- Interactive Tuning: The system can analyze a workload and generate the top-5 best physical layouts in seconds, allowing for rapid database redesign.
- Scalability: The graph-partitioning approach proved computationally feasible even for tables with hundreds of columns, solving the "layout explosion" problem.
Conclusion
HYRISE demonstrates that a flexible, automated hybrid storage engine is essential for modern main-memory databases handling mixed workloads. By prioritizing CPU cache efficiency and utilizing scalable graph-based partitioning, HYRISE provides a robust solution for real-time enterprise analytics without sacrificing transactional performance.
Summary updated