CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors

Hang Su, Zequn Liu, Chen Hu, Xuesong Lu, Yingce Xia, Zhen Liu

Abstract

While LLMs have demonstrated remarkable potential in Question Answering (QA), evaluating personalization remains a critical bottleneck. Existing paradigms predominantly rely on surface-level similarity or manual heuristics, often lacking sufficient data-driven validation. We address this by mining Community-Individual Preference Divergence (CIPD)—where individual choices override consensus—to distill six key personalization factors as evaluative dimensions. Accordingly, we introduce CoPA, a benchmark with 1,985 user profiles for fine-grained, factor-level assessment. By quantifying the alignment between model outputs and user-specific cognitive preferences inferred from interaction patterns, CoPA provides a more comprehensive and discriminative standard for evaluating personalized QA than generic metrics. The code is available at https://github.com/bjzgcai/CoPA.
BibTeX
@misc{su2026copa,
  author = {Hang Su and Zequn Liu and Chen Hu and Xuesong Lu and Yingce Xia and Zhen Liu},
  title = {{CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors}},
  year = {2026},
  eprint = {2604.14773},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.14773},
}
Figure