Multiple Information Sources Cooperative Learning

Xingquan Zhu, Ruoming JinIJCAI 2009 · OpenReview

Abstract

Many applications are facing the problem of learning from an objective dataset, whereas information from other auxiliary sources may be beneficial but cannot be integrated into the objective dataset for learning. In this paper, we propose an omni-view learning approach to enable learning from multiple data collections. The theme is to organize heterogeneous data sources into a unified table with global data view. To achieve the omni-view learning goal, we consider that the objective dataset and the auxiliary datasets share some instance-level dependency structures. We then propose a relational k-means to cluster instances in each auxiliary dataset, such that clusters can help build new features to capture correlations between the objective and auxiliary datasets. Experimental results demonstrate that omni-view learning can help build models which outperform the ones learned from the objective dataset only. Comparisons with the co-training algorithm further assert that omni-view learning provides an alternative, yet effective, way for semi-supervised learning.

Introduction

In modern data science, relevant information is often distributed across multiple heterogeneous datasets. For instance, predicting a movie's commercial success might involve a primary "Movie Table" (the objective dataset) while benefiting from external "User" or "Actor" tables (auxiliary datasets). These sources typically have different schemas and features, making traditional data integration via schema mapping labor-intensive and often impractical.

This paper introduces Omni-view Feature Learning (OFL), a framework designed to harness these disparate "islands of information" without requiring complex manual integration. By leveraging instance-level dependencies, OFL constructs a global view of the data, improving predictive performance on the primary task.

The Omni-View Feature Learning (OFL) Framework

The core of the OFL approach is to transform auxiliary data into informative features for the objective dataset. This process relies on the existence of some dependency structure between instances across datasets.

Mapping relationships between objective and auxiliary datasets.

Relational k-Means Clustering

The first stage involves a novel Relational k-Means Clustering algorithm. Unlike standard k-means, which clusters data based solely on internal features, this version considers the relationships between auxiliary instances and the objective dataset. It seeks to maximize internal similarity within clusters while minimizing the variance of their relationships to the objective data. This ensures that the resulting clusters are optimized for the target learning task.

Omni-view Feature Construction

Once the auxiliary datasets are clustered, they are used to generate new features. For each instance in the objective dataset, the framework calculates its average correlation with each auxiliary cluster. These values are then appended to the original feature set, creating a unified "omni-view" table.

The unified omni-view feature table integrating multiple information sources.

To handle new test instances that lack explicit links to auxiliary data, the framework employs a K-nearest neighbor approach in the objective feature space to estimate the corresponding omni-view features. Finally, to mitigate the "curse of dimensionality" that arises from adding numerous auxiliary features, Principal Component Analysis (PCA) is applied to compress the newly constructed feature space before training the final model.

Experimental Evaluation

The researchers evaluated OFL using 10 UCI machine learning benchmark datasets. To simulate a multi-source environment, attributes were partitioned into objective and auxiliary sets based on information gain.

Key Results

  • Performance Gains: OFL consistently outperformed Local Feature Learning (LFL) across various learners, including C4.5, Naive Bayes, and SVM. Accuracy improvements typically ranged from 1% to 15%.
  • Semi-Supervised Learning: When applied to semi-supervised tasks, OFL outperformed the classic Co-Training algorithm in 8 out of 10 datasets. The authors suggest that OFL is more robust because it introduces "attribute noise" rather than the potentially more damaging "label noise" associated with Co-Training.
  • Robustness to Incomplete Mapping: The framework remains effective even when mapping relationships between datasets are only partially known, demonstrating its practical utility in real-world scenarios where links may be sparse.

Conclusion

Omni-view Feature Learning provides a powerful alternative to traditional data integration and semi-supervised learning. By focusing on instance-level relationships rather than schema alignment, it allows models to benefit from heterogeneous auxiliary data sources. While the method depends on the existence of underlying correlations, its robustness and performance gains make it a promising approach for complex, multi-source data environments.

Summary updated

Abstract

Many applications are facing the problem of learning from an objective dataset, whereas information from other auxiliary sources may be beneficial but cannot be integrated into the objective dataset for learning. In this paper, we propose an omni-view learning approach to enable learning from multiple data collections. The theme is to organize heterogeneous data sources into a unified table with global data view. To achieve the omni-view learning goal, we consider that the objective dataset and the auxiliary datasets share some instance-level dependency structures. We then propose a relational k-means to cluster instances in each auxiliary dataset, such that clusters can help build new features to capture correlations between the objective and auxiliary datasets. Experimental results demonstrate that omni-view learning can help build models which outperform the ones learned from the objective dataset only. Comparisons with the co-training algorithm further assert that omni-view learning provides an alternative, yet effective, way for semi-supervised learning.
BibTeX
@inproceedings{zhu2009multiple,
  author = {Xingquan Zhu and Ruoming Jin},
  title = {{Multiple Information Sources Cooperative Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2009},
  url = {https://openreview.net/forum?id=S1N2C7zdWS},
}
Figure