Learning from How Users Interpret: Personalizing Recommendation Explanation Order

Deo Munduku, Elsa NegrearXiv 2026 · arXiv

TL;DR

Researchers have found that personalizing the order of sentences in a recommendation explanation—based on how a specific user has interpreted information in the past—significantly boosts their objective comprehension. In a study of 122 participants, explanations tailored to a user's unique "interpretive profile" led to an 86.5% comprehension rate, outperforming population-based averages and even a target-aware Large Language Model (LLM) approach.

What Problem Does This Paper Address?

Most personalized recommendation systems focus on what to tell you (content) or how to say it (style). For example, if you like action movies, the system highlights the stunts. However, people also differ in how they mentally organize information. One person might always look for the price first, while another might prioritize location and then check if the event is family-friendly.

The authors argue that we leave a "paper trail" of these mental habits in our written interpretations. By analyzing how a user previously described various services (like meal kits or phone plans), can we predict the best way to order information in a new explanation to make it easier to digest? The core challenge is moving beyond simple preferences to capture "interpretive regularities"—the structural habits of a user’s mind.

Methods

The researchers used the Interpretive Schema Manifestation Approach (ISMA) to build historical user profiles. They represented a user's past interpretations across three dimensions:

  1. Information Selection: Which of the six functional roles (e.g., cost, access, core attributes) did the user mention?
  2. Relative Salience: How much prominence did the user give to each role?
  3. Relational Organization (Co-mobilization): Which pieces of information did the user tend to mention together in the same sentence?

To test this, they conducted a within-participant study. First, they collected four written interpretations from each participant to build a "Historical Profile." Then, they presented new recommendations using four different sentence-ordering procedures:

  • OWN: Sentences ordered based on the user's own historical profile.
  • DONOR: Ordered based on a random other user's profile.
  • POP: Ordered based on the average of the entire population.
  • PROFILE-LLM: An LLM (Mistral-7B) was given the user's profile and the target context to determine the "best" order.

Crucially, the content and wording remained identical across all conditions; only the sequence of the six sentences changed.

Table 1: Illustrative transformation from the initial order to the final explanation order.

Datasets

The study did not use a pre-existing public dataset. Instead, the authors created a custom dataset through a two-phase experimental process:

  • Profile Construction Phase: 122 participants provided 488 written interpretations across four "source" scenarios (outdoor activity, meal-kit, mobile plan, home cleaning).
  • Evaluation Phase: The same participants were tested on four "target" scenarios (watching a film, choosing a cultural activity, a cooking workshop, and a musical performance).
Table 2: Target situations used for explanation evaluation.

Key Insights and Ideas

  • Structure is a Signal: A user's past writing reveals stable patterns in how they prioritize and group information. These patterns are "actionable signals" for UI/UX design.
  • Comprehension $\neq$ Acceptance: This is perhaps the most striking "aha moment." While personalizing the order made people understand the explanation much better, it didn't make them more likely to accept the recommendation. This suggests that better understanding doesn't automatically lead to more persuasion—it might even help a user realize a recommendation isn't right for them.
  • Rules vs. LLMs: A simple, deterministic rule based on a user's profile (OWN) outperformed a sophisticated LLM (PROFILE-LLM) that had access to the same profile plus the specific context of the recommendation. This suggests that for structural personalization, direct mapping of user habits can be more effective than "black box" reasoning.

Key Results

The primary metric was objective comprehension, measured by factual and integration questions after the explanation was shown and then removed.

  • OWN achieved 86.5% accuracy, significantly higher than all other conditions.
  • PROFILE-LLM followed at 73.0%.
  • DONOR (66.0%) and POP (63.9%) performed the worst, showing that generic or "mismatched" ordering actually hinders understanding.
Table 4: Primary objective-comprehension results. Odds ra- tios compare OWN with each alternative organization proce- dure.

The advantage of the OWN condition held true for both simple factual questions and more complex "integration" questions that required combining multiple pieces of information.

Table 5: Objective-comprehension accuracy by question type and organization condition. Values are raw proportions.

Limitations and Future Work

The authors acknowledge several boundaries to their findings:

  • Scope of Personalization: The study only varied sentence order. It didn't look at how changing the wording or adding/removing content might interact with the ordering.
  • Short-term vs. Long-term: The profiles were built from just four interpretations. It’s unclear how these interpretive habits evolve over longer periods or across very different domains.
  • Cognitive Mechanism: While comprehension improved, the study doesn't pinpoint why (e.g., reduced cognitive load, better alignment with memory structures).
  • Future Directions: The researchers suggest exploring "richer adaptive interfaces" where both content and organization are personalized, and testing these methods in real-world, long-term deployments rather than controlled lab settings.

Summary updated

Abstract

Personalized explanations typically adapt their content or presentation based on preferences, interaction histories, or other user characteristics. We examine whether regularities inferred from prior textual interpretations can support personalization of explanation organization. In a controlled within participant study with 122 participants, prior interpretations were represented along three dimensions: information selection, relative salience, and relational organization. The relational dimension was partially operationalized through co mobilization. These representations were aggregated into fixed historical profiles and used to organize otherwise identical explanation sentences. Participants encountered four organization procedures: the OWN historical profile, the DONOR historical profile, a population profile called POP, and a language model procedure using the historical profile called PROFILE LLM. Post exposure objective comprehension was the primary outcome. OWN achieved 86.5 percent correct responses, compared with 66.0 percent for DONOR, 63.9 percent for POP, and 73.0 percent for PROFILE LLM. Mixed effects analyses showed higher comprehension with OWN than with each alternative procedure. No reliable overall condition effect was detected for recommendation acceptance. These findings indicate that historical profiles derived from prior interpretations can provide an actionable signal for personalizing explanation order while keeping explanatory content and wording fixed.
BibTeX
@misc{munduku2026learning,
  author = {Deo Munduku and Elsa Negre},
  title = {{Learning from How Users Interpret: Personalizing Recommendation Explanation Order}},
  year = {2026},
  eprint = {2611.08785},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.08785},
}
Figure