Abstract
Large training datasets are required to achieve competitive performance in most natural language tasks. The acquisition process for these datasets is labor intensive, expensive, and time consuming. This process is also prone to human errors. In this work, we show that cross-cultural differences can be harnessed for natural language text classification. We present a transfer-learning framework that leverages widely-available unaligned bilingual corpora for classification tasks, using no task-specific data. Our empirical evaluation on two tasks – formality classification and sarcasm detection – shows that the cross-cultural difference between German and American English, as manifested in product review text, can be applied to achieve good performance for formality classification, while the difference between Japanese and American English can be applied to achieve good performance for sarcasm detection – both without any task-specific labeled data.
Introduction
Supervised machine learning typically relies on vast quantities of labeled data, which is often costly and slow to produce. This "data bottleneck" is particularly acute for nuanced tasks like sarcasm detection or formality classification. This research explores a novel solution: using cultural differences as a proxy for labels. By leveraging the fact that different cultures exhibit distinct linguistic signatures—such as the higher formality in German communication compared to American English—models can be trained on bilingual corpora to recognize these traits. This "cultural knowledge" is then transferred to standard English classification tasks without the need for task-specific labeled examples.
Methodology: Cross-Cultural Transfer Learning
The proposed framework utilizes a four-step pipeline to bridge cultural differences and target classification tasks:
- Transformation: Unaligned corpora from two different languages (e.g., German and American English reviews) are processed. Non-English text is translated into English to create a shared representation.
- Surrogate Training: A binary classifier is trained to distinguish the cultural origin of a document (e.g., "Originally German" vs. "Originally American").
- Target Transformation: The same transformation is applied to the target task dataset (e.g., English text for formality checking).
- Inference: The culture classifier predicts the task label. For example, text identified as "German-style" is labeled as Formal, while "American-style" text (relative to Japanese) is labeled as Sarcastic.
The authors also introduced the "LANG" document representation, a novel weighting scheme for word embeddings. This generative model highlights words frequent in one culture but rare in another, effectively filtering for the most discriminative cultural linguistic traits.
Experimental Setup
The study evaluated the framework using several datasets for training and evaluation:
- Surrogate Training: eBay Fashion (German-American) for formality and Amazon (Japanese-American) for sarcasm.
- Evaluation (Formality): Amazon Product Descriptions, New York Times snippets, and the "Formality in Online Communication" dataset (GYAFC).
- Evaluation (Sarcasm): Sarcasm Corpus V2, covering general sarcasm, rhetorical questions, and hyperbole.
Results and Discussion
The cross-cultural approach demonstrated performance remarkably close to fully supervised models. In formality classification on the Amazon Motors dataset, the transfer model achieved an AUC of 95.61, only a ~3% decrease from the supervised gold standard (98.75).
For sarcasm detection, the model reached an AUC of 76.99, within 4% of the task-specific model's 80.38. Across all tested datasets, the cross-cultural model consistently stayed within 10% of the performance of supervised gold-standards. Analysis of discriminative unigrams confirmed that the model learned stylistic "vibes"—such as the informational focus of German-style English versus the emotional focus of American-style English—rather than just topical differences.
Conclusion
This work demonstrates that culture can serve as a powerful form of distant supervision. By using cultural proxies, we can bypass the need for manually labeled data for complex linguistic tasks. Future research will focus on extending this framework to multi-class problems and exploring transformations that eliminate the need for explicit translation steps by leveraging shared cross-lingual embedding spaces.
Summary updated