Efficient Dynamic Hard Negative Sampling for Dialogue Selection

Janghoon Han, Dongkyu Lee, Joongbo Shin, Hyunkyung Bae, Jeesoo Bang, Seonghwan Kim, Stanley Jungkyu Choi, Honglak LeeACL 2024

TL;DR

This paper introduces Efficient Dynamic Hard Negative Sampling (EDHNS), a novel method to improve dialogue selection models by intelligently selecting challenging negative examples during training. EDHNS significantly boosts performance and training efficiency by filtering out easy negatives and focusing on informative pairs, making it applicable to various model architectures, including computationally intensive cross-encoders.

What Problem Does This Paper Address?

Dialogue selection tasks, common in conversational AI, involve choosing the best response or knowledge from a set of candidates. Training these models effectively requires good "negative" examples – incorrect options that help the model learn what not to pick. Traditional methods either use random negatives (which are often too easy and provide little learning signal) or static "hard" negatives (which are fixed and don't adapt to the model's learning progress). Dynamic hard negative sampling addresses this by adapting negative examples during training, but it's computationally expensive, especially for complex models like cross-encoders. This paper tackles the challenge of making dynamic hard negative sampling efficient and widely applicable.

Methods

The core contribution is the Efficient Dynamic Hard Negative Sampling (EDHNS) method, which comprises three key components:

  1. Candidate Shortlisting: This is a crucial efficiency-boosting step. Instead of evaluating all negative candidates, EDHNS first filters out "easy" negatives. It does this by defining a "confined negative candidate set" where candidates are considered "easy" if they have a low similarity score with the positive example (using a threshold τ). This significantly reduces the pool of candidates that need further processing.
  2. Hard Negative Selection: After shortlisting, the method still dynamically selects the "hardest" negatives. It computes matching scores for the remaining candidates and selects the top-l most challenging ones to use for training.
  3. Selective Update: This strategy further optimizes training by excluding question-candidate pairs where the model already exhibits high confidence. It calculates a confidence score for the positive sample and, if it exceeds a threshold, the pair is excluded from training. This prevents the model from becoming overconfident and speeds up training by focusing on pairs that still require learning.

The EDHNS framework is designed to be general and can be applied to various model architectures, including cross-encoders, which are known for their high performance but also their computational demands.

Datasets

The paper evaluates EDHNS on several standard benchmarks across two main tasks:

  • Knowledge Selection:
    • DSTC9 (Dialogue System Technology Challenge 9)
    • DSTC10 (Dialogue System Technology Challenge 10)
  • Response Selection:
    • Ubuntu Corpus V1
    • E-commerce Corpus

For DSTC10, synthetic data was constructed to address the lack of official training data, and a multi-task learning approach was used to handle speech recognition errors present in the dataset. The Ubuntu and E-commerce datasets were augmented with additional negative candidates to create a more challenging training scenario.

Key Insights and Ideas

  • Dynamic Hard Negatives are Key, but Efficiency is Crucial: The paper strongly advocates for dynamic hard negative sampling, showing it's superior to static methods. However, the real innovation lies in making it computationally feasible for complex models.
  • Filtering Easy Negatives is More Impactful Than Just Finding Hard Ones: The "Candidate Shortlisting" step is highlighted as a major contributor to efficiency. By removing the obvious wrong answers early, the subsequent "hard negative selection" can operate on a much smaller, more relevant set, leading to significant speedups without sacrificing performance.
  • Preventing Overconfidence Speeds Up Learning: The "Selective Update" mechanism, by excluding pairs the model already masters, ensures that training time is spent on genuinely informative examples, leading to faster convergence and better generalization.
  • Cross-Encoders Benefit Greatly: The paper demonstrates EDHNS's effectiveness on cross-encoder models, which are typically computationally expensive. This opens up the possibility of using advanced dynamic hard negative sampling techniques with these powerful architectures.

Key Results

EDHNS consistently and significantly outperforms baseline models across all evaluated benchmarks and tasks.

  • Knowledge Selection (DSTC9 & DSTC10): Models trained with EDHNS showed substantial improvements. For instance, on DSTC9, RoBERTa-base+EDHNS and RoBERTa-large+EDHNS saw enhancements of 4.7% and 3.2% in R@1, respectively. On DSTC10, these improvements were even more pronounced, with 6.4% and 4.5% enhancements in R@1.
  • Response Selection (Ubuntu & E-commerce): EDHNS also led to significant gains. BERT+EDHNS showed a 2.9% improvement in R@1 on Ubuntu and a remarkable 25.8% improvement on E-commerce compared to their respective baselines.
  • Efficiency Gains: Ablation studies showed that EDHNS (combining Candidate Shortlisting and Selective Update) could reduce training time by less than half while maintaining comparable performance to methods with hard negative selection alone.

Limitations and Future Work

  • False Negatives: The paper acknowledges a potential limitation is the "false negative problem," where true positives might be mislabeled as negatives in datasets (like MS MARCO). If these are mistakenly treated as hard negatives, they could negatively impact training.
  • Future Work: The authors suggest that EDHNS's effectiveness in improving both knowledge selection and response selection tasks, as well as passage reranking, highlights its generalizability. Future work could explore applying EDHNS to other NLP tasks that involve selection or ranking.

Summary updated

BibTeX
@inproceedings{han2024efficient,
  author = {Janghoon Han and Dongkyu Lee and Joongbo Shin and Hyunkyung Bae and Jeesoo Bang and Seonghwan Kim and Stanley Jungkyu Choi and Honglak Lee},
  title = {{Efficient Dynamic Hard Negative Sampling for Dialogue Selection}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure