Trained Models from "General Cross-Architecture Distillation of Pretrained Language Models into Matrix Embeddings"

Lukas Galke, Isabell Cuber, Christoph F. J. Meyer, Henrik Ferdinand Noelscher, Angelina Sonderecker, Ansgar ScherpZenodo 2022 · DOI · OpenAlex

Abstract

Trained models from the paper: Lukas Galke, Isabell Cuber, Christoph Meyer, Henrik Ferdinand Noelscher, Angelina Sonderecker, and Ansgar Scherp: <strong>General Cross-Architecture Distillation of Pretrained Language Models into Matrix Embeddings</strong>, in: <em>International Joint Conference on Neural Networks (IJCNN), </em>2022. File seq2mat_hybrid_bidirectional_sbertlike-100p-bsz512 holds the model from pretraining File ws2020_transformer_final_models holds the fine-tuned models for each task of the GLUE benchmark
BibTeX
@inproceedings{galke2022trained,
  author = {Lukas Galke and Isabell Cuber and Christoph F. J. Meyer and Henrik Ferdinand Noelscher and Angelina Sonderecker and Ansgar Scherp},
  title = {{Trained Models from "General Cross-Architecture Distillation of Pretrained Language Models into Matrix Embeddings"}},
  booktitle = {Zenodo},
  year = {2022},
  doi = {10.5281/zenodo.6533889},
}

Loading related content...

Figure