Abstract
Cross-lingual text classification is the task of assigning labels to observed documents in a label-scarce target language domain by using a prediction model trained with labeled documents from a label-rich source language domain. Cross-lingual text classification is popularly studied in natural language processing area to reduce the expensive manual annotation effort required in the target language domain. In this work, we propose a novel semi-supervised representation learning approach to address this challenging task by inducing interlingual features via semi-supervised matrix completion. To evaluate the proposed learning technique, we conduct extensive experiments on eighteen cross language sentiment classification tasks with four different languages. The empirical results demonstrate the efficacy of the proposed approach, and show it outperforms a number of related cross-lingual learning methods.
Introduction
Cross-lingual text classification (CLTC) addresses the challenge of training a model in a "label-rich" source language (e.g., English) and applying it to a "label-scarce" target language (e.g., Japanese). The fundamental hurdle is the vocabulary mismatch: an English model trained on the word "excellent" has no inherent way of knowing that the Japanese "素晴らしい" carries the same sentiment. While parallel documents are often used to bridge this gap, most existing methods learn interlingual features in a purely unsupervised way, ignoring valuable label information that could focus the model on task-relevant features.
Methodology: Semi-Supervised Matrix Completion (SSMC)
The proposed framework treats the language barrier as a missing data problem, solved through a two-step process:
- Dual-Language Matrix Construction: A document-term matrix is created where columns represent the combined vocabulary of both languages. Parallel documents have entries for both languages, while non-parallel documents have entries for only one, leaving the other half of the row "missing."
- Joint Optimization: SSMC solves an optimization problem that balances low-rank representation (via trace norm), sparsity ($L_{1,1}$ norm), and reconstruction accuracy. Crucially, it integrates a linear classification loss directly into the matrix completion process. This "semi-supervised" aspect forces the missing features to be filled in a way that maximizes discriminative power for the specific classification task.
- Dimensionality Reduction: Once the matrix is completed, Latent Semantic Indexing (LSI) is applied to extract robust, low-dimensional interlingual features for the final classifier.
Experimental Evaluation
The approach was evaluated on the Multilingual Amazon Product Review dataset, covering four languages (English, French, German, Japanese) and three categories (Books, DVD, Music). The study utilized 18 cross-lingual pairs and 2,000 unlabeled parallel document pairs per category to bridge the languages.
Results and Discussion
SSMC consistently outperformed several strong baselines, including CL-LSI (Latent Semantic Indexing), CL-KCCA (Kernel Canonical Correlation Analysis), and TSL (a two-step unsupervised matrix completion method).
As shown in Table 1, SSMC achieved superior accuracy across all 18 tasks. For example, on the English-to-French Books (EFB) task, SSMC reached 83.05% accuracy, significantly higher than CL-LSI (79.56%) and TSL (81.92%).
Beyond absolute accuracy, the method demonstrated significant label efficiency and consistency.
Figure 1 illustrates that SSMC maintains a performance lead even when very few labeled documents (100) are available in the target language. Furthermore, it shows stable improvement as more data is added, whereas some baselines like CL-OPCA occasionally exhibit performance degradation.
Conclusion and Future Work
By framing the language barrier as a matrix completion problem and incorporating labels into the feature learning process, SSMC provides a robust solution for cross-lingual classification. Future research may focus on reducing the computational complexity of the SVD-based optimization and exploring non-linear classification losses to further enhance performance.
Summary updated