The Generation Gap: Exploring Age Bias in the Value Systems of Large Language Models

Siyang Liu, Trisha Maturi, Bowen Yi, Siqi Shen, Rada Mihalcea

Abstract

We explore the alignment of values in Large Language Models (LLMs) with specific age groups, leveraging data from the World Value Survey across thirteen categories. Through a diverse set of prompts tailored to ensure response robustness, we find a general inclination of LLM values towards younger demographics, especially when compared to the US population. Although a general inclination can be observed, we also found that this inclination toward younger groups can be different across different value categories. Additionally, we explore the impact of incorporating age identity information in prompts and observe challenges in mitigating value discrepancies with different age cohorts. Our findings highlight the age bias in LLMs and provide insights for future work. Materials for our analysis will be available via url.

Introduction

While the AI community has extensively investigated biases related to race, gender, and political leaning, age bias has remained largely under-explored. As Large Language Models (LLMs) become integrated into critical sectors such as healthcare, financial advice, and daily companionship, their internal "values"—how they perceive security, religion, technology, and ethics—become increasingly significant. By 2030, nearly 45% of the U.S. population will be over the age of 45. If LLMs inherently embody the values of a younger demographic, they may struggle to provide empathetic or relevant interactions for older users, potentially alienating a massive global cohort.

Methodology

To quantify this discrepancy, we employed a multi-step analytical framework to probe the value systems of six major LLMs, including ChatGPT, Mistral, and FLAN-UL2. Our study utilizes the World Values Survey (WVS) Wave 7, a comprehensive global dataset containing responses from 94,000 individuals across 62 countries. We focused on 249 inquiries spanning 13 categories, including Social Values, Economic Values, and Religious Values.

Our analysis involved:

  1. Robust Prompting: Generating eight variations of every question to ensure results were not artifacts of specific phrasing.
  2. Value Representation: Using Principal Component Analysis (PCA) to map survey responses into a vector space, allowing us to calculate the "E-distance" between model outputs and human age groups.
  3. Trend Coefficients: Calculating the slope of the alignment gap. A positive coefficient indicates that as human age increases, the gap between the human and the AI widens.
  4. Persona Testing: Evaluating "age identity" prompting (e.g., "Suppose you are 65 years old...") to determine if models could successfully shift their value systems.

The Youthful Default

Across almost all tested models, there is a clear, statistically significant inclination toward the values of younger demographics (ages 18-24 and 25-34), particularly when compared to the U.S. population. In Western contexts (U.S., Germany, Great Britain), models showed a positive trend coefficient across nearly all categories, meaning they consistently drift away from human values as the human subjects get older.

Trend coefficients across 13 value categories for various LLMs, showing a consistent bias toward younger demographics (positive coefficients).

This bias is not uniform across all values; for instance, the gap is particularly wide in "Perceptions of Security," where older adults tend to place higher trust in established organizations—a nuance LLMs frequently miss. Furthermore, LLMs often exhibit "extreme" versions of values (e.g., being more staunchly pro-technology than any actual human age group), and these extremes almost always align with the younger end of the spectrum.

The Persona Paradox

A critical finding of our research is that simply instructing a model to adopt an older persona is often ineffective. We observed a "Persona Paradox": while a model might explicitly state it is 65 years old, its underlying logic and value judgments often remain youthful.

E-distance between model responses and human age groups with and without age-identity prompting across 13 categories.

As shown in the E-distance analysis, in 8 out of 13 value categories, adding age identity information failed to eliminate the discrepancy between the model and older cohorts. This suggests that age bias is deeply embedded in the models' training data—likely a result of the heavy use of internet-crawled data, which naturally skews toward younger, tech-savvy users.

Conclusion and Future Work

Our findings highlight a significant age-related alignment gap in current LLMs. While this bias is most pronounced in Western datasets, it suggests that LLM behavior is inextricably linked to the cultural and demographic makeup of their training data.

We advocate for the development of "age-sensitive interactions"—systems that do not merely mimic an age group through stereotyping, but instead understand and respect the diverse value systems across generations. Future research should explore more sophisticated alignment techniques, such as chain-of-thought prompting, to bridge this generational divide and ensure AI remains inclusive for all age groups.

Summary updated

Abstract

We explore the alignment of values in Large Language Models (LLMs) with specific age groups, leveraging data from the World Value Survey across thirteen categories. Through a diverse set of prompts tailored to ensure response robustness, we find a general inclination of LLM values towards younger demographics, especially when compared to the US population. Although a general inclination can be observed, we also found that this inclination toward younger groups can be different across different value categories. Additionally, we explore the impact of incorporating age identity information in prompts and observe challenges in mitigating value discrepancies with different age cohorts. Our findings highlight the age bias in LLMs and provide insights for future work. Materials for our analysis will be available via [url](https://github.com/anonymous).
BibTeX
@inproceedings{liu2024generation,
  author = {Siyang Liu and Trisha Maturi and Bowen Yi and Siqi Shen and Rada Mihalcea},
  title = {{The Generation Gap: Exploring Age Bias in the Value Systems of Large Language Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=su2PTynCil},
}
Figure