TL;DR
Vision-Language Models (VLMs) can easily be prompted to adopt a specific persona, but does an "older adult" prompt actually reflect how older people perceive the world? This paper evaluates 17 leading VLMs and finds that while they are excellent at predicting average human emotions, their age-conditioned "reasoning" relies heavily on stereotypes—often predicting more fear and negativity where real older adults actually report more positivity.
What Problem Does This Paper Address?
As VLMs are increasingly used for personalized assistants and emotional support, developers often use demographic prompts (e.g., "Imagine you are a 70-year-old") to tailor responses. However, there is a significant gap in our understanding: does following a prompt's instructions result in a model that actually captures the lived experience of that demographic?
In psychology, the "positivity effect" suggests that as people age, they tend to attend more to positive information and less to negative stimuli. This paper investigates whether VLMs capture these nuanced, age-specific shifts in perception or if they simply "hallucinate" based on common tropes about aging, such as increased frailty and nostalgia.
Methods
The researchers developed a rigorous framework to evaluate 17 VLMs (including models from OpenAI, Google, Anthropic, and Alibaba) across three dimensions:
- Competence: How well the model's baseline (no age cue) ratings match the average human rating.
- Responsiveness: How much the model's ratings change when given an age-specific prompt.
- Fidelity: Whether the direction and magnitude of the model's shift match the actual differences observed between younger and older human raters on the same images.
To ensure they weren't just measuring the model's ability to identify "pleasant" images, they used partial correlation ($r_{part}$). This statistical control allowed them to see if the model understood age-specific nuances beyond just general image pleasantness.
Datasets
- OASIS (Open Affective Standardized Image Set): The primary benchmark, containing 900 images with valence (pleasantness) and arousal (intensity) ratings from 822 humans aged 18–74.
- NAPS (Nencki Affective Picture System): Used as a secondary dataset to validate baseline competence results.
Key Insights and Ideas
- Stereotypes over Reality: Models are highly responsive to age prompts, but their shifts are poorly aligned with humans. While real older adults often rated pleasant images more positively than younger adults, VLMs predicted the opposite, lowering valence scores under the "older" persona.
- The "Sensitive Content" Reversal: The discrepancy was most jarring on sensitive but pleasant images (like erotica or nudity). Real older adults rated these more positively than younger ones, but all 17 models predicted a massive drop in pleasantness for the older persona.
- Models Agree with Each Other, Not Us: There is a high correlation ($r=0.43$) between how different models shift their ratings for age, but almost no correlation ($r=0.03$) with actual human shifts. This suggests models are drawing from a shared set of training-data biases or cultural stereotypes about the elderly.
- Language of Aging: Qualitatively, when prompted as "older," models significantly increased their use of words related to memory (nostalgia, reminiscent) and vulnerability (frail, safety, worry), regardless of whether the image actually warranted such a shift.
Key Results
- High Baseline Competence: Without age cues, models are very good at predicting average human valence (median $r = 0.891$).
- Near-Zero Fidelity: After adjusting for image pleasantness, the correlation between model and human age effects was effectively zero (ranging from $-0.030$ to $0.093$).
- Intervention Limits: The authors tested "reasoning" interventions (asking the model to debias itself) and "human-grounded" interventions (providing the model with examples of real human ratings). While providing real human data improved the direction of the shifts, image-specific fidelity remained modest ($r_{part} = 0.105$).
Limitations and Future Work
The authors acknowledge several limitations:
- Sample Size: The human reference data for the oldest group (60+) was relatively small (41 participants), creating some uncertainty in the "ground truth" magnitude.
- Demographic Scope: The study focused on an online U.S. sample; age-related affective shifts may vary across different cultures.
- Prompt Sensitivity: While they tested multiple prompt formats, the "persona" approach itself might inherently trigger stereotypical "acting" rather than genuine reasoning.
Future work should explore whether training on more diverse, demographically-labeled affective data can move models beyond "caricature" and toward true demographic alignment. For now, the paper serves as a warning: just because an AI follows a persona prompt doesn't mean it understands the person.
Summary updated