Transfer Learning Using Task-Level Features with Application to Information Retrieval

Rong Yan, Jian ZhangIJCAI 2009

Abstract

Transfer learning leverages knowledge from previous tasks to improve performance on new, data-scarce tasks. This paper introduces TL-TF (Transfer Learning with Task-Level Features), a hierarchical Bayesian approach that uses task metadata—such as query properties in Information Retrieval (IR)—to guide the transfer process. By employing a softmax gating function, TL-TF "routes" new tasks to the most relevant latent model clusters. Evaluated on TRECVID video retrieval benchmarks, TL-TF achieves up to a 39% improvement in Mean Average Precision over single-task baselines, demonstrating the power of task-level priors in multi-modal task environments.

Introduction

Traditional transfer learning models often assume that all tasks are drawn from a single, uniform distribution. However, in complex domains like Information Retrieval, tasks (queries) often form distinct clusters—for instance, searching for a person requires different features than searching for a sports event. Standard models struggle to identify which cluster a new task belongs to when training data is sparse. The authors address this by incorporating task-level features—metadata about the task itself—to inform the model's prior distribution.

Methodology

The core innovation is the TL-TF model, which operates within a hierarchical Bayesian framework.

  1. Task-Level Gating: Instead of a uniform prior, TL-TF uses a softmax gating function controlled by task-level features ($t^{(k)}$). If a query contains a person's name, the gating function increases the weight of the "person-search" model cluster.
  2. Latent Clusters: The model assumes task parameters are generated from a mixture of hidden sources. This allows the system to specialize rather than averaging across all historical data.
  3. Efficient Inference: To handle the scale of IR tasks, the authors utilize an Empirical Bayes method with variational approximation (EM algorithm with the Jaakkola-Jordan bound), making inference computationally feasible for large datasets.

Experimental Evaluation

The model was tested on the TRECVID (2002–2005) collections. The training set consisted of 88 query topics, while testing was conducted across four distinct sets (t02, t03, t04, and t05). Documents were represented by a combination of 14 high-level semantic features (e.g., "face," "outdoor") and 5 low-level retrieval experts (text, color, texture).

Key Results

TL-TF was compared against Single-Task Learning (STL) and several transfer learning baselines (TL-SC, TL-MC, TL-Comb). As shown in the results table, TL-TF consistently provided the highest performance gains across all test sets.

Performance comparison of TL-TF against baselines across TRECVID datasets.

Key findings include:

  • Significant MAP Gains: TL-TF achieved relative improvements in Mean Average Precision (MAP) of up to 39% over the STL baseline.
  • Category-Specific Success: The model excelled in "Person" and "Special Object" categories, where task-level features (like the presence of a proper noun) were highly predictive of the optimal model cluster.
  • Zero-Shot Potential: Because the gating function relies on task features known at query time, the model provides a superior starting point even before any relevance judgments are available.

Discussion: The Importance of Clustering

A critical insight from the study is that tasks are inherently multi-modal. The authors observed that performance improved significantly as the number of task clusters ($H$) increased, typically peaking around $H=6$. This confirms that "routing" a task to a specialized sub-model is more effective than using a single global transfer model.

Impact of the number of task mixtures on Mean Average Precision, comparing the proposed method (MTL-TF) to multi-cluster baselines.

Conclusion and Future Work

TL-TF demonstrates that task metadata is a powerful prior for transfer learning. While the current work relies on manually defined task features, future research could explore automated feature extraction. Additionally, extending the model to handle continuous task features could further refine the gating mechanism for even more granular task routing.

Summary updated

BibTeX
@inproceedings{yan2009transfer,
  author = {Rong Yan and Jian Zhang},
  title = {{Transfer Learning Using Task-Level Features with Application to Information Retrieval}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2009},
}
Figure