Large Language Models (LLMs) are often treated as omniscient, static encyclopedias. However, research across dozens of papers reveals that these models possess a "chaotic sense of time" Zhao et al.. Rather than treating time as a continuous dimension, LLMs function as snapshots of the specific eras most heavily represented in their training data. This leads to temporal bias: a systematic skew in performance, reasoning, and factual accuracy based on when an event occurred.
The Nostalgia Bias
One of the most counterintuitive findings in recent research is that LLMs often perform better on the distant past than on the recent past. Chenghao Zhu and colleagues term this Nostalgia Bias Zhu et al..
This occurs because historical facts—like the date of the French Revolution—are "settled." They appear in training corpora with high frequency, consistent phrasing, and little noise. In contrast, contemporary facts are often fragmented across varied reporting and linguistic noise Bajpai et al.. Consequently, models demonstrate a "temporal decay" where their reliability is tied to the volume of stable archival data rather than general reasoning capabilities Wang et al..
The Recency Effect in Fine-Tuning
While pre-training might favor the "settled" past, the process of fine-tuning introduces a conflicting Recency Bias. Yiming Ju and others have shown that the final state of a model is disproportionately shaped by the most recently observed data in a training epoch Ju et al..
This creates a "palimpsestic" effect where newer information overwrites older knowledge Kuditipudi. For example, fine-tuning on human feedback often aligns models with contemporary values, causing them to lose the historical context present in their original pre-training data Kaneko and Liu. This recency bias also manifests in recommendations, where models favor 21st-century items over relevant historical content Eberhard et al..
Failures of Temporal Grounding
LLMs struggle with temporal grounding—the ability to understand that facts change over time. A model might know that Lionel Messi played for FC Barcelona, but it may fail to realize that this fact has an expiration date Ng et al..
This lack of "temporal literacy" leads to several specific failure modes:
- Entity Substitution: Models often prioritize famous entities over the correct ones for a specific time period (e.g., naming a current president for a 1990s event) Wallat et al..
- Directional Asymmetry: Models are significantly better at "upgrading" legacy code to modern standards than "downgrading" modern code for older environments Cheng et al..
- Future Compression: Models perceive future events as more similar to one another than past events, mirroring the biological Weber-Fechner law where the distant future feels "compressed" Wang et al..
Temporal Alignment and Adaptation
To fix these biases, researchers are moving away from treating training data as a "timeless soup." Romain Fabre and colleagues demonstrated that sequential pre-training—feeding data to the model in chronological order—significantly improves "factual freshness" Fabre et al..
Other approaches include:
- Temporal Adapters: Fine-tuning models on specific "time slices" (e.g., weekly social media data) to capture the public mood of a specific moment Ahnert et al..
- Prompt Steering: Simply moving a date to the start of a query can boost temporal QA performance by up to 55% Wallat et al..
- Selective Parameter Merging: Merging models trained on different data orders to eliminate the recency bias inherent in standard fine-tuning Ju et al..
By recognizing that time is a fundamental axis of information, these researchers are transforming LLMs from static snapshots into models capable of navigating the fluid nature of human history.