Managing Storage Bloat in Edge Distributed Ledgers

16 concepts15 papers

Distributed ledger technologies (DLTs) derive their security from redundancy and immutability. Traditionally, every participating node maintains an identical, complete copy of the entire transaction history. While this design prevents single points of failure, it introduces a fundamental bottleneck when deployed to the network edge: storage bloat.

As highlighted by Youzhi Zhang in A Traceability System Based on the Quick Searching Blockchain, this "heavyweight" requirement creates an insurmountable barrier for resource-constrained devices like mobile phones, drones, and IoT sensors. An append-only ledger grows indefinitely over time. Conversely, hardware components at the edge possess finite, strict storage volumes (Kuljeet Kaur et al. in Blockchain-based Lightweight Authentication Mechanism for Vehicular Fog Infrastructure; Xianwei Wang et al. in Integration and Optimization Strategy of Blockchain-Enabled Edge Computing System for Internet of Vehicles). If a device must store gigabytes of historical transaction logs or authentication events just to verify a new transaction, it will eventually face storage exhaustion and be forced to drop out of the network (Mingyue Xie et al. in BAZAM: A Blockchain-Assisted Zero-Trust Authentication in Multi-UAV Wireless Networks).

To resolve this tension, a growing body of research focuses on architectures that allow low-power nodes to participate in consensus and verification without hosting the entire historical chain.

Decoupling and Pruning: Forgetting the Past Safely

How can a node verify the present if it throws away the past? The first core mechanism for mitigating storage bloat is decoupling data from consensus, combined with selective pruning.

Instead of forcing every node to hold full transaction payloads, architectures can separate the cryptographic proof of a transaction from its underlying data. For instance, Regio A. Michelin et al. demonstrate in SpeedyChain: A framework for decoupling data from blockchain for smart cities that decoupling transaction data from block headers allows resource-constrained edge devices to remain lean while maintaining tamper-resistance.

Once data is decoupled, nodes can safely perform garbage collection or ledger pruning—the periodic removal or archiving of expired information (Dawei xu et al. in Research Status and Prospect of Blockchain Technology in Agriculture Field; Yusen Wu et al. in Blockchains for Internet of Things: Fundamentals, Applications, and Challenges). For example, in decentralized machine learning applications, Yuzheng Li et al. show in A Blockchain-Based Decentralized Federated Learning Framework with Committee Consensus that edge nodes can delete historical blocks locally while retaining only the most recent training round's data. This creates a controllable trade-off between full historical auditability and immediate local memory constraints.

Sharding and Partitioning: Distributing the Storage Load

If individual nodes cannot hold the entire ledger, the network can divide the responsibility. Sharding and partitioning strategies ensure that the global ledger is fully preserved across the collective network, even though individual nodes only host a fraction of it.

As surveyed by Muhammad Baqer Mollah et al. in Blockchain for the Internet of Vehicles towards Intelligent Transportation Systems: A Survey, data sharding strategies minimize the local storage burden on mobile participants by distributing distinct ledger segments across different clusters of nodes. To make this partitioning secure against malicious actors, Zhikui Chen et al. introduce a reputation-based partition scheme in Reputation-based partition scheme for IoT security. By assigning nodes to specific partitions based on trust metrics, the system ensures that even though nodes operate with a minimized local storage footprint, the integrity of the partitioned ledger remains cryptographically secure.

Rateless Coding: Mathematical Redundancy Without Bloat

While partitioning divides the ledger into clean pieces, it introduces a risk: if too many nodes in a single partition go offline, that data is lost forever. To achieve both extreme storage reduction and high fault tolerance, researchers have turned to information theory—specifically, erasure coding.

Instead of storing raw blocks or simple shards, nodes can store mathematically transformed fragments of the ledger using rateless error-correcting codes. Changlin Yang et al. demonstrate this in Rateless Coded Blockchain for Dynamic IoT Networks by utilizing Raptor codes.

Original Blockchain Data
[ Block 1 ] [ Block 2 ] [ Block 3 ] [ Block 4 ]
       \         |         /         /
        v        v        v         v
             Raptor Encoding
                 /   |   \
                v    v    v
  Stream of Encoded Packets: P1, P2, P3, P4, P5, P6...

  Node A (Stores P1, P3) -> Only needs a tiny fraction
  Node B (Stores P2, P5) -> of the total data volume
  Node C (Stores P4, P6) ->

In this setup, blocks are encoded into an infinite stream of small, randomized packets. Participating IoT nodes only need to store a tiny fraction of these encoded packets. Because the code is rateless, a node can reconstruct the original ledger data by collecting any sufficient number of packets from its peers, regardless of which specific nodes are online. The authors demonstrate that this approach can reduce storage and communication overhead by up to 99.6% while maintaining near-perfect reliability in highly volatile networks.

The Trade-off: Balancing Storage and Data Availability

Optimizing for storage inevitably impacts data availability. If edge nodes aggressively prune their local storage or rely on compact cryptographic digests, the network must still guarantee that data can be retrieved and verified when needed.

Songling Fu et al. model this exact relationship in Analyzing the Impact of Storage Shortage on Data Availability in Decentralized Online Social Networks, quantifying how finite storage on mobile devices directly constrains overall data access. Furthermore, even when a client offloads the heavy storage burden to a server and keeps only a compact digest, verification remains a challenge. As Keewoo Lee notes in VeriSimplePIR: Verifiability in SimplePIR at No Online Cost for Honest Servers, storing database digests and verification proofs can still overwhelm resource-constrained devices, highlighting an ongoing frontier in minimizing client-side cryptographic overhead.

Key Foundational Work

SpeedyChain: A framework for decoupling data from blockchain for smart cities (2018) — contributed 1 concept elements to this direction.
Analyzing the Impact of Storage Shortage on Data Availability in Decentralized Online Social Networks (2014) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_distributed_ledger_storage_overhead_06c566c6,
  title = {distributed ledger storage overhead},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/distributed-ledger-storage-overhead/rdr_06c566c649224cc8b626faf8c84c8390},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_distributed_ledger_storage_overhead_06c566c6_v1,
  title = {distributed ledger storage overhead},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/distributed-ledger-storage-overhead/rdr_06c566c649224cc8b626faf8c84c8390/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}

See something wrong? Report a mistake