The Divergence of Short and Long Term Memorability

56 concepts36 papers4 proposals

When we encounter a visual stimulus—whether it is a video clip, an advertisement, or a web page—our brain forms an immediate mental representation. Traditional computational models have often treated "memorability" as a single, static attribute: a fixed score indicating how likely an image or video is to be recognized later. However, a growing body of research reveals a fundamental flaw in this assumption: what is highly memorable after three minutes is frequently forgotten after three days.

To build accurate models of human memory, researchers are shifting from static scoring to modeling temporal memorability decay. Understanding this direction requires looking beyond what we remember immediately and mapping how information filters through the brain over time.

The Asymmetry of Memory Scales

The core intuition behind temporal memorability modeling lies in a strict structural hierarchy between different timescales. As shown by Weiying Wang et al. in RUC at MediaEval 2018: Visual and Textual Features Exploration for Predicting Media Memorability, content that survives in a user's long-term memory is almost universally memorable in the short term. However, the inverse is false: immediate catchiness does not guarantee long-term retention.

This asymmetry creates a steep hurdle for standard machine learning models. While deep neural networks can easily predict short-term recognition by leveraging low-level visual salience, their performance drops precipitously when tasked with predicting long-term retention over a 24-to-72-hour window. As noted by Linli Yao et al. in RUC at MediaEval 2019: Video Memorability Prediction Based on Visual Textual and Concept Related Features, the visual and semantic cues that trigger immediate recognition are fundamentally distinct from the factors that drive multi-day storage.

[Visual Stimulus] ───> [Short-Term Memory] ───(Consolidation Filter)───> [Long-Term Memory]
                             │                                                 ▲
                             ▼                                                 │
                     (Highly Predictable via                           (Hard to Predict via
                     Low-Level Visual Features)                        Standard Visual Features)

Why Short-Term Predictors Fail

Why do immediate recall metrics fail to predict long-term persistence? The answer lies in the cognitive transition from perception to consolidation.

In Multi-Modal Ensemble Models for Predicting Video Memorability, Irving Fang demonstrates that features like audio embeddings or sudden visual shifts excel at capturing immediate attention and driving short-term recall. However, these features fail to maintain their predictive power over longer horizons because immediate recall is heavily influenced by processing fluency and attentional capture.

Over hours and days, the brain reorganizes this information. Michelangelo Naim et al. in Emergence of hierarchical organization in memory for random material observe a striking paradox: items that are easiest to recall immediately after presentation often show the poorest retention in delayed tasks if they are not integrated into a broader semantic or hierarchical context.

Neuroscientific evidence from Douglas Feitosa Tomé et al. in Dynamic and selective engrams emerge with memory consolidation explains the mechanism behind this paradox. Memory selectivity—the ability to precisely discriminate a specific memory from similar contexts—is not instantaneous. It emerges gradually over a 12-to-24-hour consolidation window through a process of "inhibitory sculpting." Without this consolidation phase, the initial neural trace simply decays into noise.

Modeling Memorability as a Continuous Decay Function

To capture this dynamic process, researchers have moved away from treating memorability as a static snapshot. Early groundwork by Aditya Khosla et al. in Memorability of Image Regions framed memory as a noisy, time-dependent degradation process, demonstrating that different image regions fade from mind at different rates.

This concept was fully realized in video modeling by Anelise Newman et al. in Multimodal Memorability: Modeling Effects of Semantics and Decay on Video Memorability. Rather than training a model to output a single memorability score, the authors designed SemanticMemNet to predict the entire mathematical decay curve of a video over time.

$$\text{Memorability}(t) = f(\text{Visual Features}, \text{Semantic Embeddings}, t)$$

By explicitly incorporating a time delay parameter $t$ into the network's objective function, the model learns how visual features interact with high-level textual semantics to resist decay. This architecture reflects the underlying cognitive reality: while raw visual salience prevents immediate decay, deep semantic meaning (such as the presence of people, actions, or narrative context) is what flattens the decay curve over days and weeks.

Real-World Implications of the Temporal Gap

Successfully separating immediate catchiness from long-term retention is critical for moving lab models into real-world applications:

  • Advertising: In Long-Term Ad Memorability: Understanding & Generating Memorable Ads, Harini S I et al. point out that the marketing industry relies entirely on multi-day retention to influence consumer behavior at the point of purchase. Models trained purely on short-term video recall fail to optimize for the lasting cognitive impact required for brand building.
  • Education and Spaced Repetition: Computational models that bridge the gap between short-term and long-term dynamics allow for highly optimized learning schedules. By analyzing initial decay rates, frameworks like the Multiscale Context Model developed by Harold Pashler et al. in Predicting the Optimal Spacing of Study: A Multiscale Context Model of Memory can accurately prescribe personalized study intervals, maximizing long-term retention over educationally relevant timescales.

By treating memory loss not as an unpredictable failure, but as a structured, feature-dependent decay function, this research direction provides the tools necessary to build systems that understand what humans will forget, what they will keep, and when.

Key Foundational Work

Neural correlates of the formation and realization of delayed intentions (2002) — contributed 1 concept elements to this direction.
Explorations with a Simulation Model of Short-Term Memory (1969) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_temporal_memorability_decay_87318fd7,
  title = {temporal memorability decay},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-memorability-decay/rdr_87318fd7d1eb45b2901034a2f39806a9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_temporal_memorability_decay_87318fd7_v1,
  title = {temporal memorability decay},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-memorability-decay/rdr_87318fd7d1eb45b2901034a2f39806a9/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake