Two Heads Are Better than One: Distilling Large Language Model Features into Small Models with Feature Decomposition and Mixture

Tianhao Fu, Xinxin Xu, Weichen Xu, Jue Chen, Ruilong Ren, Bowen Deng, Xinyu Zhao, Jian Cao, Xixin CaoAAAI 2026 · DOI

BibTeX
@inproceedings{fu2026two,
  author = {Tianhao Fu and Xinxin Xu and Weichen Xu and Jue Chen and Ruilong Ren and Bowen Deng and Xinyu Zhao and Jian Cao and Xixin Cao},
  title = {{Two Heads Are Better than One: Distilling Large Language Model Features into Small Models with Feature Decomposition and Mixture}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i23.38981},
}

Loading related content...

Figure