Characterizing Age-Conditioned Affective Reasoning in Vision-Language Models

Shahriyar Zaman Ridoy, Azmine Toushik Wasi, Lin Gu, Ruogu FangarXiv 2026 · arXiv

TL;DR

Vision-Language Models (VLMs) can easily be prompted to adopt a specific persona, but does an "older adult" prompt actually reflect how older people perceive the world? This paper evaluates 17 leading VLMs and finds that while they are excellent at predicting average human emotions, their age-conditioned "reasoning" relies heavily on stereotypes—often predicting more fear and negativity where real older adults actually report more positivity.

What Problem Does This Paper Address?

As VLMs are increasingly used for personalized assistants and emotional support, developers often use demographic prompts (e.g., "Imagine you are a 70-year-old") to tailor responses. However, there is a significant gap in our understanding: does following a prompt's instructions result in a model that actually captures the lived experience of that demographic?

In psychology, the "positivity effect" suggests that as people age, they tend to attend more to positive information and less to negative stimuli. This paper investigates whether VLMs capture these nuanced, age-specific shifts in perception or if they simply "hallucinate" based on common tropes about aging, such as increased frailty and nostalgia.

Methods

The researchers developed a rigorous framework to evaluate 17 VLMs (including models from OpenAI, Google, Anthropic, and Alibaba) across three dimensions:

  1. Competence: How well the model's baseline (no age cue) ratings match the average human rating.
  2. Responsiveness: How much the model's ratings change when given an age-specific prompt.
  3. Fidelity: Whether the direction and magnitude of the model's shift match the actual differences observed between younger and older human raters on the same images.

To ensure they weren't just measuring the model's ability to identify "pleasant" images, they used partial correlation ($r_{part}$). This statistical control allowed them to see if the model understood age-specific nuances beyond just general image pleasantness.

Figure 2. Stimulus-matched evaluation of age-conditioned affect. OASIS ratings (a) yield per-image human means ¯Hi, age effects ∆Hi, and uncertainty τi (b). 17 VLMs rate the same images under four age conditions (c), returning valence, arousal, an emotion label, and a description (d). We measure competence, responsiveness, and fidelity (rpart between ∆Mi and ∆Hi) (e), run diagnostics (f), test reasoning-only and human-grounded mitigations on 300 images (g), and validate S4 on 600 held-out images (h).

Datasets

  • OASIS (Open Affective Standardized Image Set): The primary benchmark, containing 900 images with valence (pleasantness) and arousal (intensity) ratings from 822 humans aged 18–74.
  • NAPS (Nencki Affective Picture System): Used as a secondary dataset to validate baseline competence results.

Key Insights and Ideas

  • Stereotypes over Reality: Models are highly responsive to age prompts, but their shifts are poorly aligned with humans. While real older adults often rated pleasant images more positively than younger adults, VLMs predicted the opposite, lowering valence scores under the "older" persona.
  • The "Sensitive Content" Reversal: The discrepancy was most jarring on sensitive but pleasant images (like erotica or nudity). Real older adults rated these more positively than younger ones, but all 17 models predicted a massive drop in pleasantness for the older persona.
  • Models Agree with Each Other, Not Us: There is a high correlation ($r=0.43$) between how different models shift their ratings for age, but almost no correlation ($r=0.03$) with actual human shifts. This suggests models are drawing from a shared set of training-data biases or cultural stereotypes about the elderly.
  • Language of Aging: Qualitatively, when prompted as "older," models significantly increased their use of words related to memory (nostalgia, reminiscent) and vulnerability (frail, safety, worry), regardless of whether the image actually warranted such a shift.
Figure 1. Age differences in human ratings and age-conditioned model predictions. (a) Age differences predicted by Keil and Freund [21] for each affect system; arrows give the predicted direction. (b) The same systems under the older versus young persona, pooled over 17 VLMs; shading marks whether the model change matches OASIS raters (significant, direction only, or opposite). Keil and Freund measure within-person coupling, which OASIS cannot; computed with the same across-image statistics, OASIS raters aged 60+ show no change in either coupling and rate pleasant images 0.31 points higher, a difference 16 of 17 models reverse (§G.8). (c) Per-image age effects agree across model pairs (r = 0.43) but not with humans (rpart = 0.03).

Key Results

  • High Baseline Competence: Without age cues, models are very good at predicting average human valence (median $r = 0.891$).
  • Near-Zero Fidelity: After adjusting for image pleasantness, the correlation between model and human age effects was effectively zero (ranging from $-0.030$ to $0.093$).
  • Intervention Limits: The authors tested "reasoning" interventions (asking the model to debias itself) and "human-grounded" interventions (providing the model with examples of real human ratings). While providing real human data improved the direction of the shifts, image-specific fidelity remained modest ($r_{part} = 0.105$).
Figure 3. Baseline valence, age contrasts, and agreement with human ratings. (a) Without an age cue, 17-model mean valence tracks pooled OASIS valence (median per-model r = 0.89; fitted slope 1.22; calibration slopes 1.04–1.39); sensitive-flagged images highlighted. (b) Older-minus-young valence shift with 95% intervals (also used in panels c and e): 16 of 17 models are negative, 13 after Benjamini–Hochberg correction (hollow: q ≥.05). (c) rpart with image-bootstrap intervals: point estimates lie within [−0.03, 0.09], far below the ceiling of 0.595 (dashed); the best model reaches 15.6% of it. (d) Pairwise correlations of model valence age effects: models agree with one another (mean r = 0.43 over 136 pairs) but weakly with human age effects (mean rpart = 0.03, bottom row). (e) On flagged pleasant images (n = 54), models lower valence by 0.85 on average, while older human raters score them 0.44 higher (participant bootstrap [0.10, 0.86]; 25 raters aged 60+).

Limitations and Future Work

The authors acknowledge several limitations:

  • Sample Size: The human reference data for the oldest group (60+) was relatively small (41 participants), creating some uncertainty in the "ground truth" magnitude.
  • Demographic Scope: The study focused on an online U.S. sample; age-related affective shifts may vary across different cultures.
  • Prompt Sensitivity: While they tested multiple prompt formats, the "persona" approach itself might inherently trigger stereotypical "acting" rather than genuine reasoning.

Future work should explore whether training on more diverse, demographically-labeled affective data can move models beyond "caricature" and toward true demographic alignment. For now, the paper serves as a warning: just because an AI follows a persona prompt doesn't mean it understands the person.

Summary updated

Abstract

An age prompt can change how a vision-language model (VLM) rates the emotion evoked by an image. Does the change predict how people of different ages rate that same image? We evaluate 17 VLMs on 900 OASIS images using young-, middle-, and older-adult prompts and estimate image-specific age differences from ratings by 822 participants. Without an age cue, models closely track average human valence (median Pearson $r=0.891$). Under an older-adult prompt, 16 of 17 models assign lower valence on average than under a young-adult prompt. Yet after adjusting for image pleasantness, correlations between model and human age effects range from $-0.030$ to $0.093$, against an estimated ceiling of $0.595$. The discrepancy is clearest on 54 sensitive images rated as pleasant. Across models, the older prompt reduces valence by $0.845$ points on average, while older participants rate these images $0.443$ points higher than younger participants. Providing human ratings from other images improves agreement on the direction of age differences, although image-specific fidelity remains modest on 600 held-out images ($r_{\mathrm{part}}=0.105$). These results show why evaluations of demographic personalization in foundation models need human comparisons at the level of individual stimuli. Project page available at https://shahriyar-zaman.github.io/age-conditioned-emotion-reasoning/
BibTeX
@misc{ridoy2026characterizing,
  author = {Shahriyar Zaman Ridoy and Azmine Toushik Wasi and Lin Gu and Ruogu Fang},
  title = {{Characterizing Age-Conditioned Affective Reasoning in Vision-Language Models}},
  year = {2026},
  eprint = {2611.25593},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25593},
}
Figure