For decades, the data world has been split by a fundamental trade-off. On one side are Online Transactional Processing (OLTP) systems, optimized for high-frequency, low-latency "point" updates—like swiping a credit card. On the other are Online Analytical Processing (OLAP) systems, built to scan billions of rows to find trends—like calculating quarterly revenue.
Traditionally, these two worlds are separated by a "wall." Data is born in an OLTP database and then periodically moved to an OLAP warehouse via complex Extract, Transform, Load (ETL) pipelines. As Beng Chin Ooi noted in his OTM'10 Keynote, this separation creates a performance gap and operational silos. The result is "stale" analytics; by the time data reaches the analytical engine, it no longer reflects the current state of the business.
The HTAP Convergence
The research direction known as Hybrid Transactional/Analytical Processing (HTAP) seeks to tear down this wall. The goal is to enable real-time insights on live operational data within a single system.
As highlighted by Fatma Ozcan in the VLDB 2022 Front Matter, HTAP architectures eliminate the latency associated with moving data between separate databases. By maintaining a single, consistent copy of data that serves both access patterns, organizations can perform complex queries on the most recent committed updates. Magdalena Balazinska emphasizes in the VLDB 2020 Front Matter that this convergence avoids the fragility and resource overhead of traditional ETL pipelines.
Solving the Interference Problem
The primary reason we separated these workloads in the first place was interference. A massive analytical query scanning a table can "lock" data or consume so much CPU that it stalls high-priority transactions.
To solve this, researchers focus on architectural co-design. For example, Ronald Barber and colleagues developed Wildfire, which integrates a high-performance columnar engine directly into a distributed framework like Apache Spark. This allows the system to handle ACID transactions while using open formats like Parquet for massive-scale analytics.
At the petabyte scale, systems like OceanBase Mercury, described by Quanqing Xu, use distributed architectures to balance high-concurrency writes with complex, near real-time queries. The intuition here is to decouple the storage and execution layers so that analytical throughput doesn't compromise transactional integrity.
Mechanisms for Consistency
How does a system provide a "frozen" view of data for a long-running query while transactions are constantly changing that same data? The answer lies in sophisticated snapshotting and concurrency protocols.
Liang Li and Gang Wu have explored Consistent Snapshot Algorithms specifically for main-memory databases. They introduced lightweight algorithms like Hourglass and Piggyback that outperform standard system calls (like fork()) in high-update scenarios. These mechanisms allow the analytical engine to see a consistent "point-in-time" version of the database without stopping the transactional engine from moving forward.
Expanding to Graphs and Machine Learning
The HTAP challenge isn't limited to relational tables. Emanuel Peter points out in his taxonomy of graph databases that a similar gap exists between systems optimized for local lookups (finding a friend's profile) and global analysis (running PageRank across the whole social network).
Furthermore, Beng Chin Ooi and Meihui Zhang argue for a Unified Graph Model that bridges data management and machine learning. In this view, the "analytical" part of the workload isn't just a SQL query, but the continuous training of a model on live data. The challenge remains the same: ensuring that intensive computations do not stall the real-time updates that feed them.
The Future of Unified Systems
As Xiaoou Ding and Jana Giceva note in the VLDB 2025 Front Matter, bridging the tension between these two worlds remains one of the most significant challenges in modern data management. The path forward involves "architecture-aware" designs—software co-designed with modern hardware like GPUs and many-core processors, as suggested by Bingsheng He and Jingren Zhou in their work on emerging architectures.
By unifying these workloads, we move toward a world where the distinction between "what is happening now" and "what does it mean" finally disappears.