Abstract
Effective communication relies on "common ground"—the shared cultural context and mutual knowledge between interlocutors. When this ground is missing, "pragmatic failure" occurs, leading to misunderstandings despite a shared language. This paper introduces RSA+C3, a framework that enables AI agents to dynamically infer a partner's sociocultural background and adapt their communication strategy. Evaluated on the word-association game Codenames Duet, RSA+C3 significantly improves collaborative success by modeling culture as a dynamic inference process rather than static knowledge.
Introduction
Language is more than a shared dictionary; it is a dynamic tool shaped by cultural priors. For instance, the word "boot" evokes the rear of a vehicle for a British speaker but a type of footwear for an American. Current AI models often struggle with such nuances, treating culture as fixed facts rather than a context to be inferred during interaction. This research addresses the challenge of building agents capable of overcoming pragmatic failure by modeling the listener's cultural background and updating these beliefs based on conversational feedback.
Methodology: The RSA+C3 Framework
The authors propose RSA+C3 (Rational Speech Acts for Cross-Cultural Communication), extending the standard RSA framework. In standard RSA, a pragmatic speaker selects an utterance by simulating a literal listener's interpretation. RSA+C3 introduces three key innovations:
- Multi-Cultural Modeling: The speaker maintains $n$ distinct models representing various cultural backgrounds (e.g., different nationalities or education levels).
- Bayesian Adaptation: As the interaction unfolds, the speaker observes the listener's responses and uses Bayes' rule to update the probability distribution over which cultural model best describes the partner.
- Wonky World Inference: Drawing from psycholinguistics, the model accounts for "wonky" scenarios where the partner's responses deviate from expectations, signaling a lack of shared common ground and prompting a strategy shift.
The framework was implemented using both fine-tuned word embeddings and few-shot prompted Large Language Models (LLMs) like Llama 2.
Experimental Setup
The system was evaluated using the Cultural Codes dataset, which includes 794 Codenames Duet games played by 153 human participants. These participants provided detailed demographic data, including education, political leaning, and nationality. The dataset allows for testing on "unseen" humans from diverse backgrounds, providing a robust benchmark for cross-cultural adaptation.
Results and Discussion
RSA+C3 demonstrated a significant performance boost over baseline models. In the "Education" split, guess accuracy improved from a baseline of ~43% to over 60% in some categories.
Key findings include:
- Inference vs. Labeling: The ability to infer a partner's background is more versatile than requiring explicit demographic labels, which are often unavailable in real-world settings.
- Embeddings vs. LLMs: Surprisingly, specialized word embeddings often outperformed few-shot LLMs, suggesting that deep-seated cultural associations are effectively captured through distributional semantics.
- Reasoning as a Multiplier: The pragmatic reasoning layer (RSA) is essential for translating cultural knowledge into effective, unambiguous clues.
Conclusion and Limitations
While RSA+C3 marks a significant step toward culturally aware AI, limitations remain. The Cultural Codes dataset is demographically skewed (e.g., 78% White), potentially limiting performance for underrepresented groups. Furthermore, while Codenames is an excellent proxy for referential communication, it does not capture the full complexity of natural language negotiation. Future work will explore more open-ended collaborative tasks and broader cultural datasets, while remaining vigilant against the risk of reinforcing harmful stereotypes.
Summary updated