Automatic Evaluation of Information Ordering: Kendall's Tau

Mirella Lapata

Abstract

This article considers the automatic evaluation of information ordering, a task underlying many text-based applications such as concept-to-text generation and multidocument summarization. We propose an evaluation method based on Kendall's τ, a metric of rank correlation. The method is inexpensive, robust, and representation independent. We show that Kendall's τ correlates reliably with human ratings and reading times.
BibTeX
@article{lapata2006automatic,
  author = {Mirella Lapata},
  title = {{Automatic Evaluation of Information Ordering: Kendall's Tau}},
  journal = {Comput. Linguistics},
  year = {2006},
  doi = {10.1162/coli.2006.32.4.471},
}

Loading related content...

Figure