TL;DR
This paper introduces OpenMLDB, a novel system designed for efficient and consistent real-time feature computation for online machine learning applications. It addresses the critical challenge of bridging the gap between offline model training and online model serving by providing a unified approach, significantly reducing latency and improving performance compared to existing solutions.
What Problem Does This Paper Address?
Online machine learning applications, such as fraud detection and personalized recommendations, rely heavily on real-time feature computation. Traditionally, this process is split into two distinct phases: offline for model training and online for model serving. This separation often leads to inconsistencies between the features used during training and those available in production, requiring extensive verification. Furthermore, many online ML features involve complex time-series computations and multi-table operations that existing data processing systems (like Spark and Flink) struggle to handle efficiently, leading to multi-second latencies that are unacceptable for real-time applications. The paper aims to solve these issues by providing a unified, high-performance system for both offline and online feature computation.
Methods
OpenMLDB tackles these challenges with several key innovations:
- Unified Query Plan Generator: A single query plan generator supports extended SQL syntax and compilation-level optimizations (like cycle binding and compilation caching) to ensure consistent computation results across offline and online stages, reducing deployment overhead.
- Online Real-Time Execution Engine: This engine is optimized for performance bottlenecks in ML feature computations. It uses:
- Pre-aggregation: For long window computations, intermediate aggregation results are precomputed and cached, allowing for faster lookups during online requests.
- Self-Adjusting Window Union: This technique dynamically adjusts task allocation and uses incremental computations to efficiently handle multi-table window unions, even with out-of-order data.
- Offline Batch Execution Engine: This engine is optimized for high-throughput offline computations. It features:
- Multi-Window Parallel Optimization: Allows parallel execution of independent window operations on the same table.
- Time-Aware Data Skew Resolving: Dynamically re-partitions data based on workload and distribution to mitigate data skew issues in distributed computations.
- Compact Time-Series Data Management: OpenMLDB employs a compact in-memory data format and stream-focused indexing to maximize memory usage and accelerate data access.
Datasets
The paper evaluates OpenMLDB using:
- Micro-benchmark (MicroBench): A custom tool to measure feature computation performance.
- TalkingData dataset: A publicly available dataset for evaluating memory savings.
- Akulaku's Item Ranking service (RTP) and Geographical Location Querying service (GLQ): Real-world production use cases.
- FEBench: A benchmark for real-time relational data feature extraction (mentioned in references).
Key Insights and Ideas
- Unified System is Crucial: The core insight is that a single system for both offline and online feature computation drastically reduces inconsistencies and deployment friction, saving significant engineering effort.
- Specialized Optimizations for ML Features: Standard data processing systems are not built for the unique demands of ML features (e.g., complex window functions, multi-table unions). OpenMLDB's specialized optimizations are key to achieving real-time performance.
- Pre-aggregation for Long Windows: Pre-computing parts of long window computations is a highly effective strategy to avoid re-processing massive amounts of data for every online request.
- Dynamic Load Balancing for Window Unions: Static partitioning for window unions is a bottleneck; dynamic adjustments are essential for handling varying workloads and data distributions.
- Compact Data Format Matters: Significant memory savings (over 50% compared to Spark and Trino+Redis) are achieved through a specialized in-memory data format, enabling more data to be processed in memory and reducing hardware costs.
Key Results
- Online Performance: OpenMLDB achieves 10x–20x higher online performance than Flink and DuckDB in micro-benchmarks. It also outperforms MySQL (in-memory) by over 68.4% in latency and achieves over 17x higher throughput.
- Offline Performance: OpenMLDB delivers 6x faster offline performance than Spark and MPP databases like GreenPlum.
- Memory Efficiency: OpenMLDB uses 40% less memory than in-memory databases like Redis and achieves over 50% memory saving compared to Trino+Redis.
- Reduced Latency: For real-time requests, OpenMLDB achieves latencies in the milliseconds range, crucial for online ML applications. For example, a 45x reduction in response time was observed for a specific long-window query.
- Scalability: The system demonstrates good scalability with respect to the number of threads, windows, and feature dimensions, maintaining low latency even with complex pipelines.
Limitations and Future Work
- Limitations: While not explicitly detailed as limitations, the paper focuses heavily on performance and consistency. The complexity of managing and configuring such a specialized system might be a practical consideration for users. The paper mentions that pre-aggregation introduces a small overhead during data loading.
- Future Work: The paper doesn't explicitly outline future work, but the ongoing development and community contributions (over 150 contributors, 1.6k stars on GitHub) suggest continuous improvement and expansion of features and optimizations. The system's architecture is designed to be extensible, hinting at future support for more complex ML functions and data types.
Summary updated