Reinforced Multiple Instance Selection for Speaker Attribute Prediction

Alireza Salkhordeh Ziabari, Ali Omrani, Parsa Hejabi, Preni Golazizian, Brendan Kennedy, Payam Piray, Morteza DehghaniNAACL 2024 · DOI

TL;DR

This paper introduces RL-MIL, a novel approach that frames speaker attribute prediction as a Multiple Instance Learning (MIL) problem enhanced by Reinforcement Learning (RL). RL-MIL effectively addresses the challenge of identifying relevant utterances from a large pool of speaker data, leading to improved prediction accuracy across various tasks.

What Problem Does This Paper Address?

Predicting attributes like age, gender, or political ideology from a person's language is a well-studied area. However, existing methods often struggle with the sheer volume of text a speaker might produce (e.g., social media posts, speeches). This is computationally expensive, and crucially, not all utterances are equally informative for predicting a specific attribute. Many approaches try to mitigate this by down-sampling instances, but this can lead to losing valuable information or creating sparse datasets. The paper aims to solve this by intelligently selecting the most relevant instances for prediction.

Methods

The core innovation is framing speaker attribute prediction as a Multiple Instance Learning (MIL) problem, where each "bag" is a collection of a speaker's utterances, and the "label" is the speaker's attribute. They then introduce RL-MIL, which augments standard MIL models with a Reinforcement Learning (RL) component.

Here's a breakdown:

  • MIL Framework: The problem is set up where a bag (all of a speaker's posts) is labeled, but only a subset of instances (individual posts) within that bag might be relevant.
  • RL for Instance Selection: The RL component acts as a "policy network" that learns to select a subset of instances from the bag. This selection is guided by a reward signal, which is the performance of the downstream MIL model on the selected instances. The goal is to maximize this reward, effectively learning to pick the most informative utterances.
  • Model-Agnostic: RL-MIL is designed to be compatible with various MIL pooling methods (Mean, Max, Attention, Rep-The-Set) and can also be used with ensemble MIL models.
  • Loss Function: A custom loss function is used to train the RL policy, encouraging it to select instances that lead to higher prediction scores (F1-score) and penalizing it for selecting instances that lead to lower scores.

Datasets

The authors evaluated their approach on a combination of real-world and synthetic datasets:

  • Real-world Datasets:
    • Congressional Speeches: Used for predicting gender, political ideology, and age.
  • Facebook + MFQ Data: Used for predicting moral foundations (care, fairness, loyalty, authority, purity).
  • Synthetic Datasets:
    • Toxic-10 and Toxic-5: Derived from the "Civil Comments" dataset, these were created to specifically control the number of toxic instances in a bag, allowing for a controlled study of signal sparsity.

Key Insights and Ideas

  • Intelligent Selection is Key: The paper strongly suggests that simply feeding all instances into a MIL model is suboptimal. Learning to select the most relevant instances significantly boosts performance, especially when the signal is sparse.
  • RL as a Powerful Selector: Reinforcement Learning proves to be an effective mechanism for learning this instance selection policy, outperforming heuristic or random selection methods.
  • MIL is a Natural Fit: The MIL formulation is well-suited for speaker attribute prediction, as it naturally handles the nested structure of data (speaker -> utterances).
  • Signal Sparsity Matters: The performance gains of RL-MIL are particularly pronounced in scenarios where only a small fraction of utterances are indicative of the target attribute, as demonstrated by the synthetic datasets.

Key Results

  • Significant Improvement: RL-MIL consistently outperformed baseline MIL and ensemble MIL approaches across various tasks and datasets.
  • Real-world Performance: On average, the best RL-MIL variant was 5% better than the best MIL variant and 3.1% better than the best ensemble-MIL variant in macro F1 score.
  • Synthetic Data Gains: On the synthetic Toxic-10 and Toxic-5 datasets, RL-MIL showed substantial improvements, outperforming other methods by 5.7% and 12.8% respectively. This highlights its effectiveness in handling sparse signals.
  • Attention Pooling Synergy: The Attention pooling method, when combined with RL-MIL, often yielded the best results, suggesting a strong synergy between learned instance weighting and learned instance selection.

Limitations and Future Work

  • Computational Bottlenecks: The RL component, while powerful, can introduce computational overhead, potentially limiting its use in real-time applications or resource-constrained environments.
  • RL Optimization Challenges: Fine-tuning RL hyperparameters and ensuring model robustness were noted as challenges.
  • Data Bias: The reliance on English text and the potential for biases in training data could limit generalization to other languages or diverse populations.
  • Future Work: The authors suggest exploring alternative RL designs, optimizing the integration of RL and MIL, and extending the approach to other MIL problems with sparse signals (e.g., clinical reports). They also highlight the potential for RL-MIL to provide exemplar-based explanations.

Summary updated

BibTeX
@inproceedings{ziabari2024reinforced,
  author = {Alireza Salkhordeh Ziabari and Ali Omrani and Parsa Hejabi and Preni Golazizian and Brendan Kennedy and Payam Piray and Morteza Dehghani},
  title = {{Reinforced Multiple Instance Selection for Speaker Attribute Prediction}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.181},
}
Figure