Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration

Zhongzhi Yu, Zheng Wang, Yonggan Fu, Huihong Shi, Khalid Shaikh, Yingyan Celine Lin

BibTeX
@inproceedings{yu2024unveiling,
  author = {Zhongzhi Yu and Zheng Wang and Yonggan Fu and Huihong Shi and Khalid Shaikh and Yingyan Celine Lin},
  title = {{Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=DLTjFFiuUJ},
}
Figure