Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors
Lists
Only you see this
BibTeX
@inproceedings{amos2024never,
author = {Ido Amos and Jonathan Berant and Ankit Gupta},
title = {{Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors}},
booktitle = {International Conference on Learning Representations},
year = {2024},
url = {https://openreview.net/forum?id=PdaPky8MUn},
}| Long Range Arena : A Benchmark for Efficient Transformers | arXiv · 2020 |
| Block-Biased Mamba for Long-Range Sequence Processing | arXiv · 2025 |
| On the Long Range Abilities of Transformers | arXiv · 2023 |
| Simplifying and Understanding State Space Models with Diagonal Linear RNNs | arXiv · 2022 |
| Spectral State Space Models | arXiv · 2023 |
| Pretraining Without Attention | EMNLP · 2023 |