Distribution Kernel Methods for Multiple-Instance Learning

Gary DoranAAAI 2013 · DOI

Abstract

Multiple-Instance Learning (MIL) is traditionally treated as a problem of learning from sets of points. This research proposes a shift in perspective, reframing MIL as a problem of learning from distributions. By utilizing kernel mean maps and introducing a "kernel median map," we develop a framework that is more robust to noise and theoretically consistent with the underlying generative processes of physical data. This approach is particularly relevant for complex tasks such as 3D-QSAR in drug discovery and aerosol remote sensing in climate science.

Introduction: The Multiple-Instance Learning Problem

Multiple-Instance Learning (MIL) is a supervised learning paradigm where training data consists of "bags" of instances rather than individual labeled points. A bag is assigned a positive label if it contains at least one positive instance; it is labeled negative only if all instances within it are negative.

A quintessential application of MIL is 3D-QSAR (Quantitative Structure-Activity Relationship) in drug discovery. Here, a molecule represents a bag, and its various 3D conformations represent the instances. A molecule is considered "active" if at least one of its conformations can bind to a specific target protein receptor.

Illustration of Multiple-Instance Learning in 3D-QSAR. Molecule A is active because one of its conformations fits the receptor, while Molecule B is inactive as none of its conformations match.

Challenges in Current MIL Approaches

The author identifies two primary limitations in existing MIL methodologies:

  1. Neglect of Generative Processes: Most algorithms treat bags as arbitrary sets, ignoring that instances often arise from an underlying physical distribution, such as the Gibbs free energy distribution governing molecular shapes.
  2. The Soundness Gap: Many "set kernels" can mathematically separate bag labels without respecting the logical MIL assumption. This can lead to models that achieve low bag-level error but are inconsistent with the true instance-level labels.

Methodology: Distributional Representations

To address these issues, this research explores the use of Distribution Kernels. Instead of analyzing individual points, these methods map an entire distribution into a high-dimensional Reproducing Kernel Hilbert Space (RKHS).

Kernel Mean and Median Maps

  • Kernel Mean Map: This provides an injective mapping that represents a distribution as a single point in a feature space. The empirical version, the Average-Normalized Set Kernel, is already a staple in some MIL literature.
  • Kernel Median Map: A key contribution of this work is the proposal of the kernel median map. By using the spatial median in the RKHS instead of the mean, the representation becomes significantly more robust to "noisy" or outlier instances, which are prevalent in small bags.

Applications and Results

The framework is applied to two critical domains:

  • 3D-QSAR: Predicting biological activity based on molecular conformations.
  • Climate Science: Handling uncertainty in pixel-level data for satellite-based aerosol remote sensing.

Preliminary results indicate that distribution-based set kernels can outperform state-of-the-art MIL baselines, particularly in regression tasks where real-valued activity levels are predicted. Furthermore, this research establishes a theoretical link between distribution-based learning and the PAC (Probably Approximately Correct) learning framework for MIL.

Conclusion and Future Work

Reframing MIL through distribution kernels provides a theoretically sound and robust framework for learning from complex, structured data. Future research will focus on the theoretical characterization of the kernel median map, methods for recovering instance-level labels from bag-level distributions, and extending these techniques to other structured objects like trees and graphs.

Summary updated

BibTeX
@inproceedings{doran2013distribution,
  author = {Gary Doran},
  title = {{Distribution Kernel Methods for Multiple-Instance Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2013},
  doi = {10.1609/aaai.v27i1.8501},
}
Figure