Multiview Semi-supervised Learning for Ranking Multilingual Documents

Nicolas Usunier, Massih-Reza Amini, Cyril Goutte

Abstract

In Information Retrieval (IR), the scarcity of labeled data across multiple languages poses a significant challenge for "learning to rank" systems. This paper introduces SmVR (Semi-supervised Multiview Ranking), a novel algorithm that treats different language translations of a document as separate "views." By enforcing a consensus on the relative ordering of unlabeled documents across these views, SmVR effectively regularizes the learning process. Experimental results on the Reuters RCV1/RCV2 corpus demonstrate that SmVR consistently outperforms supervised baselines and classification-oriented semi-supervised methods, especially in scenarios with high class imbalance.

Introduction

Modern search engines and document retrieval systems must operate in increasingly multilingual environments. However, the standard "learning to rank" paradigm requires extensive manual labeling—a process that is both expensive and time-consuming when scaled across multiple languages. Furthermore, most existing semi-supervised approaches are designed for classification rather than ranking. While classification focuses on absolute labels (relevant vs. irrelevant), ranking focuses on the relative order of documents, which is more robust to the class imbalance typically found in retrieval tasks.

Methodology: The SmVR Algorithm

The proposed SmVR algorithm bridges the gap between multiview learning and semi-supervised ranking by leveraging the inherent redundancy in multilingual corpora.

Multiview Framework

The core assumption is that a single document can be represented in multiple "views"—in this case, its translations into different languages (e.g., English, French, German). Each view provides a distinct feature set for the same underlying semantic content.

The Agreement Principle and Unanimity Rule

SmVR relies on the principle that rankers trained on different views should agree on the relative ordering of unlabeled documents. The algorithm follows an iterative pseudo-labeling process:

  1. Initialization: Train initial rankers for each language using a small set of labeled documents.
  2. Pairwise Comparison: Evaluate pairs of unlabeled documents across all view-specific rankers.
  3. Unanimity Constraint: If all rankers agree that Document A is more relevant than Document B, this pair is added to the training set as a "pseudo-labeled" example.
  4. Iteration: Retrain the rankers using the expanded dataset and repeat.

To maintain computational efficiency and avoid the $O(m^2)$ complexity of comparing all unlabeled pairs, the authors employ random sampling of pairs (e.g., 15,000 pairs per iteration). The underlying base learner is a linear Support Vector Machine tailored for ranking (SVM-multi).

Experimental Evaluation

The authors evaluated SmVR using a multilingual sub-collection of the Reuters RCV1/RCV2 corpus, comprising 111,740 documents available in five languages: English, French, German, Italian, and Spanish. Six topic categories were used to define relevance for bipartite ranking tasks.

Performance Gains

SmVR demonstrated consistent superiority over both fully supervised baselines and single-view semi-supervised models. For instance, in the "M11" topic category, SmVR achieved an Average Precision (AvP) of 0.727, a substantial improvement over the 0.566 achieved by the standard supervised baseline.

Robustness to Data Scarcity

A key finding was the algorithm's efficiency in low-data regimes. The multiview approach with only 10 labeled instances achieved a reduction in disagreement on unlabeled data comparable to a single-view model trained on 200 labeled instances. This 20x improvement in data efficiency highlights the power of consensus as a regularization mechanism.

Discussion: Ranking vs. Classification

The study explicitly compared SmVR against a multiview classification approach (SmVC-SVM). The results confirmed that ranking-specific models are better suited for IR tasks. Ranking models optimize for the area under the ROC curve (AUC), making them naturally more robust to imbalanced datasets where relevant documents are rare. SmVR maintained high performance even when the training set contained as few as 1 or 2 positive examples out of 10.

Conclusion and Future Work

SmVR provides a robust framework for multilingual document ranking when labeled data is scarce. By leveraging the inherent redundancy in multilingual corpora, it achieves significant performance gains over traditional methods.

Future research may address the strictness of the "unanimity" rule, which could become a bottleneck as the number of views increases. Exploring "soft" consensus or majority voting could improve scalability. Additionally, extending the framework from bipartite ranking to multi-level relevance scales and ad-hoc search scenarios remains a promising direction.

Summary updated

Abstract

We address the problem of learning to rank documents in a multilingual context, when reference ranking information is only partially available. We propose a multiview learning approach to this semi-supervised ranking task, where the translation of a document in a given language is considered as a view of the document. Although both multiview and semi-supervised learning of classifiers have been studied extensively in recent years, their application to the problem of ranking has received much less attention. We describe a semi-supervised multiview ranking algorithm that exploits a global agreement between view-specific ranking functions on a set of unlabeled observations. We show that our proposed algorithm achieves significant improvements over both semi-supervised multiview classification and semi-supervised single-view rankers on a large multilingual collection of Reuters news covering 5 languages. Our experiments also suggest that our approach is most effective when few labeled documents are available and the classes are imbalanced.
BibTeX
@inproceedings{usunier2011multiview,
  author = {Nicolas Usunier and Massih-Reza Amini and Cyril Goutte},
  title = {{Multiview Semi-supervised Learning for Ranking Multilingual Documents}},
  booktitle = {ECML/PKDD},
  year = {2011},
  doi = {10.1007/978-3-642-23808-6\_29},
}
Figure