Human Interpretation and Exploitation of Self-attention Patterns in Transformers : A Case Study in Extractive Summarization

Raymond Li, Wen Xiao, Lanjun Wang, Giuseppe Carenini

Abstract

The transformer multi-head self-attention mechanism has been thoroughly investigated recently. On one hand, researchers are interested in understanding why and how transformers work. On the other hand, they propose new attention augmentation methods to make transformers more accurate, efficient and interpretable. In this paper, we synergize these two lines of research in a human-in-the-loop pipeline to first find important task-specific attention patterns. Then those patterns are applied, not only to the original model, but also to smaller models, as a human-guided knowledge distillation process. The benefits of our pipeline are demonstrated in a case study with the extractive summarization task. After finding three meaningful attention patterns in the popular BERTSum model, experiments indicate that when we inject such patterns, both the original and the smaller model show improvements in performance and arguably interpretability.
BibTeX
@inproceedings{li2022human,
  author = {Raymond Li and Wen Xiao and Lanjun Wang and Giuseppe Carenini},
  title = {{Human Interpretation and Exploitation of Self-attention Patterns in Transformers : A Case Study in Extractive Summarization}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.emnlp-main.694},
}
Figure