Software Testing and the Naturally Occurring Data Assumption in Natural Language Processing

K. Bretonnel Cohen, William A. Baumgartner Jr., Lawrence HunterSETQALNLP 2008

Introduction

In Natural Language Processing (NLP), a common dogma persists: the belief that "naturally occurring data" (large corpora) is the gold standard for testing systems. This paper challenges that assumption, demonstrating that a small, hand-crafted suite of functional tests exercises significantly more of a system's logic than a massive corpus of nearly 4 million words.

The Problem with "Real-World" Data

In academic NLP, researchers often assume that if a system performs well on a large, "real-world" dataset, it is well-tested. The authors argue that this "naturally occurring data assumption" is a dangerous fallacy. High performance on a corpus doesn't mean the code is bug-free; it just means the bugs weren't triggered by that specific data. Relying solely on corpora leaves large portions of code—and potential "showstopper" bugs—completely unexercised.

To illustrate the stakes, the authors point to the cautionary tale of Geoffrey Chang, a scientist whose career was derailed when a simple sign error in his code led to the retraction of five high-profile papers. In NLP, software testing is often neglected in favor of performance metrics like F1 scores, which can mask underlying software defects.

Methodology

The researchers used code coverage—a standard software engineering metric—to evaluate testing effectiveness. They measured three specific types:

  • Line coverage: The proportion of individual lines of code executed.
  • Branch coverage: Whether the code has traversed both the "true" and "false" paths of conditional statements (like if/else).
  • Class coverage: The number of programming classes touched during the run.

They applied these metrics to OpenDMAP, an information extraction system, comparing the coverage achieved by a massive natural corpus (3.9 million words from BioCreative II) against a suite of 278 "functional tests"—short, synthetic inputs designed to target specific features and edge cases.

The Plateau of Diminishing Returns

A striking finding of the study is the rapid plateauing of coverage when using natural data. After processing just 1,000 sentences, adding more data to the corpus had virtually zero effect on code coverage. Even though the corpus contained millions of words, it repeatedly hit the same code paths while missing others entirely.

Line coverage plateaus quickly as the number of sentences increases, demonstrating diminishing returns.

Naturally occurring data is effective for testing the "happy path" (common linguistic patterns), but it rarely triggers error-handling code or complex logic designed for rare edge cases. By using coverage analysis, the authors identified "untested" sections of code and immediately uncovered two "showstopper" bugs that the massive corpus had never triggered.

Comparative Results

The quantitative gap between structured testing and corpus-based testing was substantial. The small suite of 278 functional tests achieved higher coverage across every metric compared to the 4-million-word corpus.

Comparison of coverage metrics between functional tests and the large corpus.

As shown in the results, the functional tests covered 56% of the code, while the full corpus only reached 41%. In the specific part of the code that handles extraction rules, the functional tests achieved 100% class coverage and 71% branch coverage, compared to the corpus's 90% and 24%, respectively.

Conclusion

While 100% coverage is a high bar, the 50-60% range achieved by functional tests provides a much safer baseline than the ~40% achieved by the corpus alone. The authors acknowledge that code coverage is not a silver bullet—it cannot detect "errors of omission" where a feature was never implemented. However, the study suggests that a combination of structured functional tests and natural corpora provides the most robust toolkit for ensuring the reliability of NLP applications.

Summary updated

BibTeX
@inproceedings{cohen2008software,
  author = {K. Bretonnel Cohen and William A. Baumgartner Jr. and Lawrence Hunter},
  title = {{Software Testing and the Naturally Occurring Data Assumption in Natural Language Processing}},
  booktitle = {SETQALNLP},
  year = {2008},
}
Figure