Attending via both Fine-tuning and Compressing

Jie Zhou, Yuanbin Wu, Qin Chen, Xuanjing Huang, Liang He

Abstract

This paper introduces a novel "learner-compressor" framework designed to enhance the interpretability and performance of attention mechanisms in neural networks. By leveraging the Information Bottleneck (IB) principle, the framework "purifies" attention scores, forcing models to ignore noisy, irrelevant input and focus on the true drivers of a decision. Through iterative phases of fine-tuning for representation and compression for relevance, the proposed Variational Information Bottleneck ATtention (VAT) mechanism achieves superior classification accuracy and more faithful model explanations across multiple benchmark datasets without requiring human-labeled rationales.

Introduction

In Deep Learning, the Attention Mechanism is frequently utilized as a proxy for model interpretability. The underlying assumption is that high attention weights correlate with the features most influential to a prediction. However, recent critiques have questioned this assumption, noting that standard attention often focuses on irrelevant "noise"—for instance, highlighting the word "movie" instead of "funny" in a sentiment analysis task. While training on human-provided rationales can mitigate this, such data is expensive and labor-intensive to collect. This research addresses the challenge by automatically forcing attention to be more faithful and interpretable using only standard task labels.

The Learner-Compressor Framework

The authors propose a dual-phase iterative framework that separates the learning of high-quality representations from the refinement of attention weights:

  1. The Learner: In this phase, a standard attention-based model (e.g., LSTM or BERT) is fine-tuned on a specific task. To ensure the encoder focuses purely on representation, the attention parameters are held fixed.
  2. The Compressor: This phase introduces the Variational Information Bottleneck ATtention (VAT) mechanism. Grounded in Information Theory, the compressor aims to map the text representation into a latent variable $Z$ that maximizes information about the output label $Y$ while minimizing mutual information with the input $X$.
The Variational Information Bottleneck Attention (VAT) architecture, illustrating the compression of representation R into latent variable Z.

By "squeezing" the information flow, the compressor naturally discards redundant or noisy features. The framework iterates between these two phases, allowing the learner to benefit from "purified" attention while the compressor refines its bottleneck based on improved representations.

Experimental Evaluation

The framework was evaluated across eight diverse text classification benchmarks, including sentiment analysis (IMDB, SST, Yelp, Twitter), topic classification (AG News, TREC), and subjectivity detection (Subj). These datasets range from short-form tweets to long-form reviews, testing the framework's robustness across varying text lengths and domains.

Key Results and Discussion

Performance Gains

The integration of VAT consistently improved classification performance. Across all eight datasets, the VAT-enhanced versions of LSTM and BERT outperformed their standard counterparts. For example, BERT-VAT achieved an average accuracy of 87.27%, a notable improvement over the 86.52% achieved by standard BERT-ATT.

Faithfulness and Rationale Discovery

A critical finding is that interpretability does not have to come at the cost of performance. The authors utilized the Area Over the Perturbation Curve (AOPC) metric to measure faithfulness—how much model accuracy drops when "important" words are removed. The VAT models demonstrated significantly higher AOPC scores, indicating that the words they attended to were truly critical for the prediction.

Rationale extraction accuracy on the IMDB dataset for LSTM and BERT models, comparing standard attention (ATT) with the proposed VAT framework.

Furthermore, as shown in the rationale extraction results, the VAT framework significantly outperforms standard attention in identifying the most relevant words (Top K) that align with human-labeled rationales, even in a zero-shot setting where no rationale labels were provided during training.

Conclusion

The learner-compressor framework demonstrates that interpretability can be framed as an information-theoretic optimization problem. By treating attention as a bottleneck, the model naturally filters out "fluff," leading to both higher accuracy and more reliable explanations. While currently focused on text classification and local attention, future work may extend this principle to multi-head attention structures and other domains such as Computer Vision.

Summary updated

BibTeX
@inproceedings{zhou2021attending,
  author = {Jie Zhou and Yuanbin Wu and Qin Chen and Xuanjing Huang and Liang He},
  title = {{Attending via both Fine-tuning and Compressing}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.189},
}
Figure