Decoupling Logical Data Abstractions from Physical Storage

57 concepts56 papers1 proposal

In the early days of software engineering, application logic was tightly bound to the physical layout of data. If a database schema changed, an indexing strategy was updated, or a file format evolved, the application code interacting with that data inevitably broke. To build scalable, maintainable, and resilient systems, computer scientists developed a powerful architectural pattern: logical-physical data decoupling.

By inserting an intentional, mediated abstraction layer between how data is conceptually understood (the what) and how it is physically stored and accessed (the how), systems allow both application logic and storage infrastructure to evolve independently.

The Problem of Structural Rigidity

When software routines directly query or manipulate raw storage structures, they become brittle. A change in the underlying data format forces a cascading rewrite of the application code. As Pradeeban Kathiravelu et al. note in their work on Data Services with Bindaas: RESTful Interfaces for Diverse Data Sources, hard-coded queries and custom API layers tightly couple business logic to specific database implementations, severely hindering migration and maintenance.

This rigidity was recognized early in the evolution of computer science. In the 1970s, Robert Balzer argued in CASAP: A Testbed for Program Flexibility that software must remain agnostic of underlying database structures. He proposed a mediated information retrieval layer that allows routines to request data dynamically based on intent rather than requiring hard-coded parameters. Similarly, Erik Sandewall argued in Ideas About Management Of LISP Data Bases that treating a knowledge base as a persistent entity independent of specific software routines ensures that data remains a reusable asset across multiple applications.

The Core Mechanism: Intentional vs. Extensional Layers

The fundamental mechanism behind this decoupling is the separation of the data architecture into two distinct layers:

  1. The Intentional (Logical) Layer: Contains the conceptual models, logical axioms, views, and user-facing query interfaces.
  2. The Extensional (Physical) Layer: Manages raw data storage, memory layouts, file formats, and indexing mechanisms.

Koichi Furukawa formalized this distinction in A Deductive Question Answering System on Relational Data Bases, demonstrating that separating logical views from physical storage allows developers to modify the underlying database schema without altering the user's conceptual query model.

+--------------------------------------------------------+
|                Intentional (Logical) Layer             |
|   (Logical Schemas, Abstract Data Types, User Queries) |
+--------------------------------------------------------+
                            |
                            v  [Mediation / Translation Layer]
                            |
+--------------------------------------------------------+
|                Extensional (Physical) Layer            |
|   (Indexes, Materialized Views, Specific DB Engines)   |
+--------------------------------------------------------+

To make this work, the system must automatically map logical requests to physical execution plans. As Alin Deutsch et al. explain in Physical Data Independence, Constraints, and Optimization with Universal Plans, true physical data independence requires that users write queries against a logical schema without needing to understand underlying storage mechanisms like indexes or materialized views. The system's optimization layer handles the translation, ensuring that physical changes do not break query validity or efficiency.

This mapping mechanism becomes especially critical in heterogeneous environments. For instance, Tim Mattson et al. developed the BigDAWG polystore system (Demonstrating the BigDAWG Polystore System for Ocean Metagenomics Analysis), which uses abstract data model layers to decouple the user's query interface from diverse underlying storage engines (relational, array, and text), routing queries to the optimal native engine transparently.

Compilation and Mediation Mechanics

How do these abstraction layers operate without introducing prohibitive performance overhead? The literature highlights two primary strategies: automatic compilation and runtime mediation.

Automatic Compilation

Instead of interpreting abstractions at runtime, some systems compile high-level logical specifications directly into optimized physical structures. Donald Cohen demonstrated this in Automatic Compilation of Logical Specifications into Efficient Programs via the AP5 compiler, which provides a uniform relational interface while allowing the underlying storage to utilize diverse, efficient data structures based on performance annotations.

A similar approach was taken by Gordon S. Novak with GLISP (GLISP: A High-Level Language for A.I. Programming), which decouples program logic from data representations, compiling abstract expressions into high-performance machine code regardless of whether a property is a stored value or a dynamically computed result.

Runtime Mediation

Other architectures employ a dedicated middleware or service layer to intercept and translate requests. In blackboard AI architectures, Daniel D. Corkill et al. designed GBB (GBB: A Generic Blackboard Development System), which decouples the logical definition of multi-dimensional data structures from their physical storage and indexing implementations. This separation allows developers to tune data retrieval strategies independently without modifying high-level application logic (Daniel D. Corkill et al., Achieving Flexibility, Efficiency, and Generality in Blackboard Architectures).

In modern web service architectures, this mediation is often achieved via RESTful abstractions or service layers. Yaqing Si et al. demonstrate in An Open Web Service-Based DSS how a data service abstraction layer can replace direct database queries with service calls, ensuring decision logic remains independent of physical data structures.

Why Decoupling Enables System Evolution

By breaking the hard dependency between application logic and physical storage, decoupled architectures unlock several critical capabilities:

Ultimately, decoupling works because it acknowledges that application logic and physical storage operate on different lifecycles. By formalizing the boundary between them, systems gain the flexibility to adapt to changing operational demands without sacrificing structural integrity.

Key Foundational Work

CASAP: A Testbed for Program Flexibility (1973) — contributed 2 concept elements to this direction.
Ideas About Management Of LISP Data Bases (1975) — contributed 1 concept elements to this direction.
CASAP: A Testbed for Program Flexibility (1973) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_architectural_decoupling_2bf630d1,
  title = {architectural decoupling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/architectural-decoupling/rdr_2bf630d185e044c3be541f7e77f9744d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_architectural_decoupling_2bf630d1_v1,
  title = {architectural decoupling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/architectural-decoupling/rdr_2bf630d185e044c3be541f7e77f9744d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake