Echoes of the Prior: A Computational Phenomenology of Forgetting

Gege Gao, Bernhard Schölkopf, Andreas Geiger

TL;DR

Echoes of the Prior is an interactive art installation that models the subjective experience of neurodegeneration and forgetting by inducing controlled decay within a feed-forward 3D reconstruction network. By degrading sensory weights and twisting semantic memory manifolds, the system provides a real-time, visceral visualization of a "silicon brain" losing its structural grip on reality.

What Problem Does This Paper Address?

While neuroscience can map the physical mechanisms of neurodegeneration—such as synaptic pruning, plaques, and tangles—it cannot convey the internal, subjective experience of a fading mind. We know that a patient forgets, but we cannot easily visualize how the world looks to them as it dissolves.

Furthermore, most contemporary AI art focuses on the genesis of intelligence (learning and generation). This paper flips the paradigm to explore the entropy of intelligence (forgetting). The authors bridge computer vision, computational psychiatry, and media arts to create a structural analogy for cognitive decline, transforming an invisible internal process into a shared, visceral visual medium.

Methods

The authors build a real-time interactive installation called DecArt (a portmanteau of Decay and Art). The system uses a feed-forward 3D Gaussian Splatting (FF-3DGS) framework backed by Depth Anything 3 to reconstruct scenes in real time. To simulate cognitive decline, they intercept the network's processing pathways using two parallel decay functions governed by a user-controlled decay factor ($\lambda \in [0, 1]$):

  1. Sensory Decay ($G_{sense}$): Acting as a proxy for the sensory cortex, a shallow CNN processes raw pixels. The authors perturb its weights using three biologically inspired mechanisms:
  2. Spectral Amnesia: Dampens high-frequency components of convolutional kernels via Fast Fourier Transforms (FFT) to simulate the loss of fine-detail acuity.
  3. Neural Fabrication: Injects spatially smoothed Gaussian noise into the weights to mimic the spontaneous, uninhibited neural activity that causes hallucinations (e.g., Charles Bonnet Syndrome).
  4. Structural Atrophy: Applies a stochastic mask to zero out weight connections and decay biases, creating a sparse, ghostly 3D reconstruction.
  5. Memory Decay ($F_{mem}$): Acting as a proxy for long-term semantic memory, a DINOv2 foundation model extracts high-dimensional latent concepts. The authors introduce cognitive confusion by applying a continuous orthogonal rotation matrix $R(\lambda)$ to the latent space. This "twists" the semantic manifold—misaligning feature directions while preserving vector norms—and combines it with stochastic noise to simulate the breakdown of top-down predictive priors.
Fig. 1. Echoes of the Prior. A computational simulation of the fragility of memory under entropic loss.
  1. Selective Forgetting: By integrating Grounded-SAM, users can input a text prompt (e.g., "woman in red") to isolate specific objects. The system performs "latent space surgery," applying the decay function exclusively to tokens associated with that object, causing it to ghost out of existence while the surrounding environment remains intact.

Datasets

Because Echoes of the Prior is an interactive art installation rather than a standard benchmarking paper, it does not evaluate performance on traditional computer vision datasets. Instead, the system supports dual inputs:

  • A Curated Archive of semantic archetypes for immediate exhibition engagement.
  • A Personal Injection feature that allows users to upload their own photos or videos via AirDrop or QR code, transforming the installation into a personalized digital elegy.

For latent space validation, the authors analyze how distinct semantic concepts cluster within the DINOv2 space using standard object classes.

t-SNE visualization of the DINOv2 latent space showing distinct manifolds of semantic concepts forming a structured, healthy memory baseline.

Key Insights and Ideas

  • Deep Glitch vs. Surface Glitch: Unlike traditional glitch art that operates on surface pixels (e.g., datamoshing), DecArt introduces structural pathology. By eroding internal network weights and predictive priors, the resulting artifacts emulate true neurodegeneration rather than simple signal transmission errors.
  • Volumetric Memory Space: The authors argue that volumetric 3D representation (via Gaussian Splatting) is fundamentally closer to the phenomenology of biological memory than flat 2D imagery. Human memory encoding is deeply intertwined with spatial navigation in the hippocampus; therefore, to remember is to inhabit a space.
  • The Computational Uncanny Valley: At intermediate decay levels ($0.3 < \lambda < 0.7$), the network enters a regime where structural integrity is partially preserved but semantic alignment is twisted. Objects retain their textures but undergo topological metamorphoses (e.g., a solid building melting like a liquid). This computationally mirrors Associative Agnosia and Freud's concept of das Unheimliche (the familiar-yet-wrong).
  • Ontology of the Shell: The authors deliberately chose monocular depth estimation (which projects a fragile, hollow 2.5D shell) over watertight 3D generative models. When a viewer rotates the scene, the hollow shell exposes catastrophic voids behind near-coherent facades, serving as a powerful metaphor for the incomplete, fragile nature of memory.

Key Results

The system successfully achieves a closed-loop, real-time interactive performance pipeline running at 60 FPS. As a user physically manipulates a hardware fader or digital slider to adjust $\lambda$, the 3D scene responds instantly without inference latency.

Qualitatively, the installation shifts from a pristine, photorealistic reconstruction ($\lambda = 0$) to a subtle shimmer ($\lambda \approx 0.1$), through an unsettling zone of topological distortion ($0.3 < \lambda < 0.7$), and finally to total semantic dissolution and oblivion ($\lambda > 0.8$).

Fig. 2. Installation Design of Echoes of the Prior. Snapshot of the real-time feedback (𝜆= 0.21). The viewport displays the immediate structural dissolution of the 3D scene as the user adjusts the decay factor.

Limitations and Future Work

  • Artistic Analogy, Not Mechanism: The authors emphasize that the mappings between artificial neural networks and biological processes (e.g., CNN weights to V1 simple cells) are productive artistic metaphors, not claims of literal mechanistic equivalence.
  • The Hard Problem of Consciousness: While the system successfully simulates the look of forgetting, it cannot simulate the feeling of loss or sadness; the machine is simply processing noise. Future work could integrate Affective Computing modules to modulate decay based on a simulated emotional state.
  • Simplified Cognitive Pathway: The current framework bypasses crucial intermediate cognitive stages such as attention and working memory, modeling forgetting solely as a direct decay of sensory processing and semantic priors.
  • Transition to Spiking Neural Networks (SNNs): To better simulate living memory, the authors aim to move away from static Artificial Neural Networks toward event-driven SNNs, where forgetting can be visualized as the actual silencing of time and the cessation of neural firing.

Summary updated

Abstract

Memory is not merely the storage of data; it is the scaffolding of reality. When biological memory fades, the world does not simply turn black; it regresses into an unrecognizable chaos. Echoes of the Prior is an interactive installation that attempts to visualize this subjective phenomenology of forgetting. By inducing controlled synaptic decay within a Feed-Forward 3D Reconstruction model, we create an artistic analogy for the erosion of the brain's predictive priors. We position the Neural Network not as a tool for engineering, but as a cognitive proxy - a silicon brain whose structural degeneration evokes the disorienting, poetic, and terrifying experience of losing one's grip on the world. Ultimately, we offer this framework as a catalyst, inviting the wider community to explore the uncharted potential of neuromorphic aesthetics in visualizing the fragility of intelligence. Interactive demo see https://decart-4d.github.io/.
BibTeX
@article{gao2026echoes,
  author = {Gege Gao and Bernhard Schölkopf and Andreas Geiger},
  title = {{Echoes of the Prior: A Computational Phenomenology of Forgetting}},
  journal = {Proc. ACM Comput. Graph. Interact. Tech.},
  year = {2026},
  doi = {10.1145/3816072},
}
Figure