Multi-view clustering of multilingual documents

Young-Min Kim, Massih-Reza Amini, Cyril Goutte, Patrick GallinariSIGIR 2010 · DOI

Introduction

In the contemporary global information environment, data is frequently represented across multiple "views"—distinct sets of features describing the same underlying entity. A classic instance is a multilingual news corpus, where a single event is documented in several languages. The challenge in multi-view clustering is to effectively synthesize these diverse representations into a single, coherent topical structure.

Traditional methodologies generally fall into two categories: Early Fusion, which involves the simple concatenation of feature vectors from all views, and Late Fusion, which reconciles independent clusterings performed on each view. Early fusion often suffers from noise and the high dimensionality of combined feature spaces, while late fusion may fail to capture the latent synergies between views during the initial learning process. This paper introduces voted-PLSA, a novel two-stage framework that utilizes independent view "votes" to establish a robust global consensus.

Methodology: The voted-PLSA Framework

The voted-PLSA approach transforms an unsupervised clustering task into a semi-supervised one by identifying high-confidence document assignments across multiple views.

Stage I: Independent View Voting

The process begins by applying Probabilistic Latent Semantic Analysis (PLSA) to each language view independently. For every document, the model generates a "voting pattern"—a sequence of topic assignments derived from each language. For example, in a five-language setup, a document might receive a pattern such as $(T_1, T_1, T_2, T_1, T_1)$, where $T_n$ represents a specific latent topic.

Stage II: Constrained Multi-view Clustering

The second stage leverages these voting patterns to guide the final clustering:

  1. Signature Identification: The algorithm identifies the most frequent voting patterns, designated as "cluster signatures."
  2. Pre-assignment: Documents that align with a signature (allowing for a $V-1$ match threshold) are considered "easy" cases and are pre-assigned to their respective clusters.
  3. Constrained PLSA: A final global PLSA model is executed on the concatenated features of the entire corpus. During the EM update steps, the assignments of the pre-assigned documents are held constant. These "anchor" documents provide the necessary supervision to resolve the cluster assignments of more ambiguous, "difficult" documents.

Experimental Setup

The researchers evaluated the framework using a multilingual corpus derived from the Reuters RCV1/RCV2 dataset. The dataset contains over 110,000 documents translated into five languages: English, French, German, Italian, and Spanish. The documents are distributed across six distinct thematic categories.

Results and Analysis

The experimental results demonstrate that voted-PLSA consistently outperforms baseline methods across key metrics, including micro-averaged precision and Normalized Mutual Information (NMI).

Performance comparison of different clustering strategies.

As shown in the table above, voted-PLSA achieved a precision of 0.65, compared to 0.63 for feature concatenation (conc-PLSA) and 0.61 for late fusion (Fusion-LM). These improvements were found to be statistically significant.

The success of the method is rooted in the high quality of the initial "voting" stage. As detailed in the following table, the first stage successfully pre-assigned approximately 62.84% of the documents with an average precision of 0.76.

Stage I pre-assignment statistics across languages.

Conclusion

The voted-PLSA framework provides a robust middle ground between early and late fusion strategies. By treating independent language models as voters, the system effectively identifies a reliable consensus that anchors the global clustering process. While the current implementation assumes a fixed number of topics across views, future research could explore adaptive topic counts and the integration of other generative models like Latent Dirichlet Allocation (LDA).

Summary updated

BibTeX
@inproceedings{kim2010multi,
  author = {Young-Min Kim and Massih-Reza Amini and Cyril Goutte and Patrick Gallinari},
  title = {{Multi-view clustering of multilingual documents}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2010},
  doi = {10.1145/1835449.1835633},
}
Figure