TL;DR
Researchers from Amazon Alexa AI and academic partners demonstrate that user personality traits—specifically Agreeableness—significantly bias how people rate their interactions with conversational agents. By embedding a personality survey within a fictional story told by the agent, the team found that "agreeable" users are essentially "easy graders," providing higher ratings regardless of the agent's actual performance.
What Problem Does This Paper Address?
In the world of Spoken Dialogue Systems (SDS), user ratings (like a 1-to-5 star scale) are the gold standard for measuring success. Typically, developers average these ratings across all users to decide if a model update was "good."
However, this approach relies on a flawed assumption: that all users use the scale the same way. In reality, a "3-star" rating from a grumpy, introverted user might mean something entirely different than a "3-star" rating from a highly agreeable social butterfly. If we don't account for these individual differences, we risk optimizing our AI for an "average user" who doesn't actually exist, potentially baking sociodemographic and personality biases into our systems. The challenge is: how do you measure a user's personality without boring them with a 50-question psychology survey in the middle of a chat?
Methods
The authors propose a "Story Method" to elicit personality traits naturally.
- The Story-Based Survey: Instead of a traditional questionnaire, the agent tells a story about going to a party. At specific turns, it asks the user questions like, "I just love going to parties, what about you?" (targeting Extraversion) or "Would you have gone to check on [a sad person]?" (targeting Agreeableness).
- Validation: Before deploying it, they validated this method on Amazon Mechanical Turk (mTurk) by having users take both the "story survey" and a standard Big Five personality marker test. They used Cronbach’s Alpha to check internal consistency and linear regression to see if story responses could predict standard survey scores.
- Deployment: They integrated the story into an experimental conversational agent. The story was implemented as a Finite State Machine (FSM) to ensure every user had the exact same experience during the personality-probing phase, even though the rest of the agent was neural-network-based.
Datasets
- Pilot/Validation Set: Data from 96 participants on Amazon Mechanical Turk used to correlate the story method with the standard Big Five Factor Markers.
- Experimental Set: Interactions with 760 real-world users. Out of these, 307 users provided a final conversation rating, and 233 of those engaged with the personality story enough to be included in the final analysis.
Key Insights and Ideas
- Agreeable People are "Easy Graders": The most significant finding was that Agreeableness is a strong predictor of higher ratings ($\beta = 0.316, p = 0.02$). This suggests that some of the "improvement" we see in AI models might just be noise caused by the personality distribution of the testers.
- The "Opt-in" Boost: Users who chose to hear the story rated the agent significantly higher (3.63 vs. 3.0) than those who declined. This points to a selection bias: people who are open to engagement and storytelling are predisposed to like the agent more.
- Extraversion is Complicated: While Extraversion correlates with how humans rate other humans, its effect on rating AI was only nearing significance ($p = 0.098$). The authors suggest this might be because conversational agents usually "lead" the chat, meaning an AI partner doesn't need to be "talkative" or "confident" in the same way a human friend does to satisfy an extrovert.
- Mitigating Social Desirability Bias: By framing questions within a story, users may be less likely to give "the right answer" and more likely to give an honest one, as their focus is on the narrative rather than a formal assessment.
Key Results
- Validation Success: The story scores were statistically significant predictors of traditional survey scores for both traits ($p < 0.01$).
- Rating Bias: Agreeableness and Extraversion together explained a small but significant portion of the variance in conversational ratings ($R^2 = 0.045, p = 0.012$).
- Engagement: 70% of users agreed to hear the story, proving that "gamifying" personality collection is a viable way to get data without ruining the user experience.
Limitations and Future Work
The authors acknowledge a few hurdles:
- Sample Skew: The study saw a "ceiling effect" for Agreeableness—most people who opted into the story were already quite agreeable, making it harder to see the full spectrum of behavior.
- Low R-Squared: While personality is a factor, the $R^2$ of 0.045 means 95% of the rating variance is still caused by other things (like the agent's actual intelligence, latency, or the user's mood).
- Static Dialogue: The story was a fixed script. Future work could involve more dynamic, personalized storytelling that adjusts to the user's previous answers while still maintaining enough consistency for scientific measurement.
Ultimately, this paper serves as a wake-up call for NLP researchers: if you aren't looking at who is giving the ratings, you're only seeing half the picture.
Summary updated