GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow
Lists
Only you see this
Abstract
GPT-Neo is an implementation of model & data-parallel GPT-2 and GPT-3-like models, utilizing Mesh Tensorflow for distributed support. This codebase is designed for TPUs. It should also work on GPUs, though we do not recommend this hardware configuration.
BibTeX
@inproceedings{black2021gpt,
author = {Sid Black and Leo Gao and Phil Wang and Connor Leahy and Stella Biderman},
title = {{GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow}},
booktitle = {Zenodo},
year = {2021},
doi = {10.5281/zenodo.5297715},
}Loading related content...