On Combining Side Information and Unlabeled Data for Heterogeneous Multi-Task Metric Learning

Yong Luo, Yonggang Wen, Dacheng Tao

Introduction

Distance Metric Learning (DML) is a fundamental component in machine learning, essential for tasks such as $k$-nearest neighbor classification and clustering. The goal is to find a mathematical measure that accurately reflects the "closeness" between data points. Traditionally, this requires significant "side information"—labels indicating which pairs of items are similar or dissimilar. However, real-world applications often face two major hurdles: data scarcity in target domains and heterogeneity, where different domains use entirely different feature representations (e.g., SIFT features vs. Wavelet textures, or different languages).

Existing solutions for heterogeneous transfer are often limited to pairwise relationships between two domains, ignoring the richer "high-order" correlations available when multiple domains are considered simultaneously. To address these challenges, the authors propose Heterogeneous Multi-Task Metric Learning (HMTML).

The HMTML Framework

HMTML treats metric learning as a multi-task problem across $M$ heterogeneous domains. The core innovation lies in how it bridges these disparate feature spaces using both limited labeled data and abundant unlabeled data.

Conceptual overview of the HMTML framework showing the integration of labeled side information and unlabeled data across heterogeneous domains to learn a shared latent subspace.

Key Components

  • Metric Decomposition: The framework decomposes the metric for each domain ($A_m$) into a transformation matrix ($U_m$). This matrix maps high-dimensional, domain-specific features into a lower-dimensional common subspace.
  • High-Order Tensor Correlation: Instead of simple pairwise comparisons, HMTML utilizes unlabeled data—samples that exist across all domains—to construct a high-order covariance tensor. By maximizing the correlation across all domains simultaneously in the shared subspace, the model learns a more reliable transformation than pairwise methods.
  • Optimization and Constraints: The objective function incorporates a non-differentiable hinge loss for labeled pairs, solved efficiently using Nesterov’s smoothing technique and a Projected Gradient Method (PGM). The model also enforces sparsity and non-negativity constraints to ensure only relevant features are used and natural correlations are preserved.

Experimental Evaluation

The model was evaluated on two distinct heterogeneous datasets:

  1. Reuters Multilingual Collection (RMLC): A text dataset where domains are defined by language (English, Italian, and Spanish), resulting in varying TF-IDF vector dimensions.
  2. NUS-WIDE (Animal Subset): An image dataset where domains are defined by different feature extraction techniques: Bag of Visual Words (BOVW), Color Autocorrelogram (CORR), and Wavelet Texture (WT).

Key Results

HMTML was compared against standard Euclidean distance (EU), single-domain metric learning (RDML), and state-of-the-art heterogeneous transfer methods like DAMA and MTDA.

Classification accuracy of HMTML compared to baselines (EU, RDML, DAMA, MTDA) across different numbers of labeled samples (5, 10, 15) and varying common factor dimensions.

As shown in the experimental results, HMTML consistently outperforms competitive methods, particularly when labeled data is scarce. On the RMLC dataset, HMTML outperformed MTDA by 16.8% with only 5 labeled samples per category. In the image annotation task, HMTML maintained robust performance even when feature types were radically different, whereas methods like DAMA struggled significantly. The experiments also indicated that the best results typically occur at a relatively low dimensionality (around 30 factors for text), suggesting the model effectively distills data into core concepts.

Conclusion and Future Work

HMTML demonstrates that leveraging unlabeled data as a bridge through high-order tensor correlations can significantly enhance metric learning in heterogeneous environments. While highly effective, the current framework relies on linear transformations. Future research directions include extending the method to learn nonlinear metrics and optimizing the computational complexity of high-order tensor operations for even larger-scale multi-domain tasks.

Summary updated

Abstract

Distance metric learning (DML) is critical for a wide variety of machine learning algorithms and pattern recognition applications. Transfer metric learning (TML) leverages the side information (e.g., similar/dissimilar constraints over pairs of samples) from related domains to help the target metric learning (with limited information). Current TML tools usually assume that different domains exploit the same feature representation, and thus are not applicable to tasks where data are drawn from heterogeneous domains. Heterogeneous transfer learning approaches handle heterogeneous domains by usually learning feature transformations across different domains. The learned transformation can be used to derive a metric, but these approaches are mostly limited by their capability of only handling two domains. This motivates the proposed heterogeneous multi-task metric learning (HMTML) framework for handling multiple domains by combining side information and unlabeled data. Specifically, HMTML learns the metrics for all different domains simultaneously by maximizing their high-order correlation (parameterized by feature covariance of unlabeled data) in a common subspace, which is induced by the transformations derived from the metrics. Extensive experiments on both multi-language text categorization and multi-view social image annotation demonstrate the effectiveness of the proposed method.
BibTeX
@inproceedings{luo2016combining,
  author = {Yong Luo and Yonggang Wen and Dacheng Tao},
  title = {{On Combining Side Information and Unlabeled Data for Heterogeneous Multi-Task Metric Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure