Designing LLMs for cultural sensitivity: Evidence from English-Japanese translation

Helene Tenzer, Oumnia Abidi, Stefan FeuerriegelarXiv 2025 · DOI · arXiv

TL;DR

Large Language Models (LLMs) are increasingly used for cross-cultural communication, but their ability to translate with cultural sensitivity remains a challenge. This paper demonstrates that simple prompt engineering, by specifying the target audience's cultural background, can significantly improve the cultural appropriateness and user perception of LLM-generated translations, even for languages with distinct communication norms like English and Japanese.

What Problem Does This Paper Address?

As LLMs become integrated into everyday communication tools, particularly in multilingual workplaces, ensuring their translations are not just linguistically accurate but also culturally appropriate is crucial. Literal translations can lead to misunderstandings, misinterpretations, and damaged trust due to differences in politeness, directness, and communication styles across cultures. While LLMs excel at literal accuracy, their ability to adapt to culture-specific language patterns, especially in high-stakes professional contexts like workplace emails, is not well understood. This paper investigates whether and how LLMs can be guided to produce culturally sensitive translations, specifically focusing on English-to-Japanese workplace emails.

Methods

The study employed a mixed-methods approach:

  • Qualitative Analysis (Study 1): The researchers used a cross-cultural speech act coding scheme (House & Kádár) to analyze linguistic patterns in translated emails. They manually coded translations for formality, directness, and the use of mitigating language. This allowed them to assess how well LLM translations adapted to Japanese culture-specific language patterns under different prompting strategies.
  • User Evaluation (Study 2): A within-subjects online experiment was conducted with 50 native Japanese speakers. Participants rated the perceived appropriateness and their intention to comply with email requests based on translations generated using three different prompt strategies.

Three prompting strategies were tested:

  1. Naïve Prompt: A simple "Translate to Japanese" instruction.
  2. Audience-Targeted Prompt: Specified the recipient's cultural background (Japanese supervisor) and requested a polite, natural tone.
  3. Instructional Prompt: Included explicit guidance on Japanese business etiquette, forms of address, and communication norms.

Datasets

The study used a corpus of five hypothetical U.S. English workplace emails, designed to reflect high-stakes, socially delicate interactions involving face-threatening speech acts (request, refusal, disagreement, apology, gratitude). These emails were translated into Japanese using GPT-4.5 under the three different prompting strategies, resulting in a dataset of 15 Japanese translations. No new benchmarks were introduced; instead, the focus was on evaluating existing LLM capabilities on a carefully constructed corpus.

Key Insights and Ideas

  • Literal Accuracy Isn't Enough: The study strongly suggests that LLMs' impressive literal translation capabilities do not automatically translate to cultural appropriateness.
  • Prompt Engineering is Powerful: Even minimal prompt engineering, like specifying the audience's cultural background, can significantly improve the cultural sensitivity of LLM translations.
  • Explicit Guidance Yields Best Results: Providing explicit instructions on cultural norms and etiquette leads to the most culturally appropriate translations, though the incremental gain over audience-targeted prompts can be modest.
  • Cultural Nuances Matter in Professional Settings: Differences in formality, directness, and politeness are critical in professional communication and can be effectively addressed through tailored prompts.
  • User Perception Aligns with Linguistic Analysis: The qualitative linguistic analysis of translations closely mirrored the quantitative findings from user evaluations, validating the effectiveness of the prompt strategies.

Key Results

  • Qualitative Analysis:
    • The naïve prompt resulted in only superficial adjustments towards Japanese cultural norms.
  • The audience-targeted prompt showed a clear shift towards Japanese patterns.
  • The instructional prompt yielded the most extensive adaptation across pragmatic categories like alerters, speech act perspective, upgraders, and supportive moves.
  • Level of directness showed little systematic change, while downgraders exhibited a more nuanced pattern.
  • User Evaluation:
    • Perceived Appropriateness: Translations from the audience-targeted prompt were rated significantly more appropriate than naïve translations, and instructional prompts further increased this perception. The mean appropriateness score increased from 5.52 (naïve) to 5.72 (audience-targeted) to 5.83 (instructional).
  • Intention to Comply: The instructional prompt significantly boosted the intention to comply with email requests compared to the naïve prompt. Mean intention to comply scores were 5.41 (naïve), 5.55 (audience-targeted), and 5.64 (instructional).

Limitations and Future Work

  • Small Corpus: The study's corpus was small due to the complexity of the multi-method coding. Future work should explore a broader range of text genres (e.g., instant messaging, meeting minutes).
  • Single LLM: The analysis focused on GPT-4.5. Future research should compare multiple LLMs to assess the generalizability of the findings.
  • English-Japanese Focus: The study concentrated on a high-resource language pair. Investigating low-resource language pairs is crucial to understand the limits of simple prompting strategies.
  • Self-Reported Data: Study 2 relied on self-reported perceptions and intentions. Field experiments in live communication settings would provide stronger ecological evidence.

The authors suggest that future research should encourage the HCI community to treat cultural sensitivity as a crucial quality criterion for LLM technology and advocate for LLM interfaces that offer easy "culture-aware" options.

Summary updated

BibTeX
@misc{tenzer2025designing,
  author = {Helene Tenzer and Oumnia Abidi and Stefan Feuerriegel},
  title = {{Designing LLMs for cultural sensitivity: Evidence from English-Japanese translation}},
  year = {2025},
  eprint = {2509.11921},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.11921},
}
Figure