AMALGUM -- A Free, Balanced, Multilayer English Web Corpus

Luke Gessler, Siyao Peng, Yang Liu, Yilun Zhu, Shabnam Behzad, Amir Zeldes

Abstract

We present a freely available, genre-balanced English web corpus totaling 4M tokens and featuring a large number of high-quality automatic annotation layers, including dependency trees, non-named entity annotations, coreference resolution, and discourse trees in Rhetorical Structure Theory. By tapping open online data sources the corpus is meant to offer a more sizable alternative to smaller manually created annotated data sets, while avoiding pitfalls such as imbalanced or unknown composition, licensing problems, and low-quality natural language processing. We harness knowledge from multiple annotation layers in order to achieve a "better than NLP" benchmark and evaluate the accuracy of the resulting resource.
BibTeX
@misc{gessler2020amalgum,
  author = {Luke Gessler and Siyao Peng and Yang Liu and Yilun Zhu and Shabnam Behzad and Amir Zeldes},
  title = {{AMALGUM -- A Free, Balanced, Multilayer English Web Corpus}},
  year = {2020},
  eprint = {2006.10677},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2006.10677},
}
Figure