Temporal Relevance Aware Cache Replacement

52 concepts40 papers1 proposal

Traditional caching strategies rely on simple heuristics like Least Recently Used (LRU) or Least Frequently Used (LFU). These algorithms operate on the assumption that the past is a direct prologue to the future: if a data item was accessed recently or frequently, it will likely be accessed again soon.

However, in dynamic environments—such as live web agents, streaming media networks, and rolling machine learning pipelines—data items have explicit, finite lifetimes or time-varying utility. A piece of data might be highly popular right now but completely irrelevant in five minutes. To address this, a growing body of research focuses on temporal relevance-aware cache replacement, where eviction decisions are guided by explicit models of item lifetimes, temporal utility decay, and predictive time-domain allocation.

The Intuition of Lifetime-Based Eviction

Why look at remaining lifetime instead of past recency? Imagine a cache filled with content that expires at different rates, such as real-time social media feeds or volatile sensor data. An LRU policy might keep an item that was accessed one second ago, even if its data becomes stale and useless in the next second.

To maximize cache utility under these conditions, György et al. (2017) proposed the Longest lifetime In–Shortest lifetime Out (LISO) policy. Instead of tracking access history, LISO evaluates the remaining relevance duration of content. By swapping out items with the shortest remaining lifetime for those with the longest, the system maximizes the probability that cached content remains valid when a user eventually requests it. As the authors later demonstrated in György et al. (2018), analyzing the full distribution of item lifetimes within the cache allows systems to make optimal trade-offs between proactive data fetching and future data relevance.

Time-Domain Buffer Sharing and Predictive Eviction

Traditional caches are reactive: they evict an item only when a new item arrives and space is needed. However, if we can predict when an item will be needed next, we can treat cache storage as a resource shared explicitly across the time domain.

Chen et al. (2018) formalized this concept as Time Domain Buffer Sharing. Instead of holding onto an item until it is forced out by a capacity miss, the system calculates an optimal duration for a file to occupy memory based on predicted request distributions. Once an item's peak utility period passes, it is proactively evicted. This shifts the paradigm from managing cache space to managing cache time, balancing hit ratios against the average memory footprint. Similarly, Al-Abbasi et al. (2019) introduced tunable time windows to standard LRU policies in multi-tier video streaming networks, optimizing eviction windows based on content popularity and edge storage constraints.

Reactive Caching (LRU):
[ Item A (Fresh) ] -> [ Item B (Fresh) ] -> [ Item C (Stale but Recent) ] -> Evict oldest when full

Time-Domain Caching:
[ Item A (Peak Utility Window) ] -> Proactive Eviction -> [ Free Space Allocated to Item B ]

Balancing Recency and Frequency with Temporal Decay

When explicit lifetimes are unknown, systems must infer temporal relevance by combining recency and frequency with mathematical decay functions. Pure frequency counts suffer from "cache pollution," where historically popular items never get evicted even after they become obsolete.

To solve this, Zhang et al. (2016) utilized Modified Simple Exponential Smoothing (MSES) to create a time-aware, frequency-based algorithm that continuously discounts older accesses. In hardware and industrial applications, Arribas García et al. (2023) implemented an aging mechanism that periodically decays performance counters. This ensures the cache prioritizes recent memory access behavior over stale historical data, allowing rapid adaptation during program phase transitions. For resource-constrained AI agents, Yu et al. (2024) leveraged the Window Tiny Least Frequently Used (W-TinyLFU) policy to manage short-term memory, filtering out noisy or outdated data over long execution sequences.

Preserving Temporal Diversity

In some advanced applications, the goal is not just to keep the most recent items, but to maintain a diverse temporal distribution of items. If a cache only keeps the absolute newest entries, it risks a catastrophic failure mode where a cluster of items expires simultaneously, leaving the cache empty.

In the context of streaming multi-armed bandits, Braverman et al. (2026) showed that recency-prioritized buffer management must deliberately distribute stored candidates across time to avoid simultaneous expiration.

A particularly creative solution to this problem was designed by Xia et al. (2024) for machine learning model unlearning on edge devices. The authors used a Fibonacci sequence to determine which historical model checkpoints to replace. This creates a non-linear jumping pattern in the cache, preserving a wide, diverse temporal distribution of historical states rather than just the most recent ones. This ensures that a relevant historical checkpoint is always available when an unlearning request occurs, demonstrating that thinking about time non-linearly can drastically improve cache utility.

Key Foundational Work

Abstract planning in dynamic environments (2001) — contributed 3 concept elements to this direction.
The Intelligent Database Interface: Integrating AI and Database Systems (1990) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_lifetime_based_cache_replacement_578937b6,
  title = {lifetime-based cache replacement},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/lifetime-based-cache-replacement/rdr_578937b6481c4df1af9b20664e1488b8},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_lifetime_based_cache_replacement_578937b6_v1,
  title = {lifetime-based cache replacement},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/lifetime-based-cache-replacement/rdr_578937b6481c4df1af9b20664e1488b8/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake