TL;DR
This paper quantifies how much more repetitive contract language is compared to general English text. By analyzing various linguistic features, they found contracts exhibit significantly less vocabulary diversity and higher sentence similarity, suggesting that contract language, while still natural, operates within a more constrained linguistic space. This has implications for NLP tools designed to process legal documents.
What Problem Does This Paper Address?
The common perception is that legal contracts are "repetitive." While this is often attributed to drafting practices like using precedents, the paper aims to move beyond anecdotal evidence and quantify how repetitive contract language is. Specifically, it investigates whether contract language is statistically more repetitive than other forms of English text, using standard corpus linguistics metrics. This is important because understanding this repetition can inform the development of more effective Natural Language Processing (NLP) tools for legal applications, as well as provide insights into the linguistic properties of this specialized domain.
Methods
The authors employ a multi-faceted approach to analyze repetition:
- Token-level Analysis (Zipfian Analysis): They analyze the distribution of word frequencies using log-log plots (rank-vs-counts). This method, based on Zipf's Law, helps reveal vocabulary richness. A steeper curve indicates fewer rare words and thus higher repetition.
- Hapax Legomena Analysis: They specifically examine "hapax legomena" – words that appear only once in a corpus. A higher ratio of hapax legomena to total tokens suggests greater vocabulary diversity.
- Pronoun Frequency Analysis: They analyze the frequency of pronouns, which are common words in general language but can be used differently or less frequently in specialized texts.
- Sentence-level Similarity (Nearest Neighbors Analysis): Using a unigram vector model and cosine similarity, they measure how similar each sentence is to its closest match within the corpus. This assesses repetition at a discourse level.
Datasets
The paper uses the following datasets:
- Contract Corpora: A custom corpus of 1,737 contracts, categorized into:
- Non-Disclosure Agreements (NDAs)
- Prime Contracts
- Purchase Orders
- Service Agreements
- Subcontracts
- Baseline Corpora:
- Wikipedia (English): A large corpus derived from Wikipedia articles.
- Brown Corpus: A well-established corpus of American English text from the mid-20th century.
The authors pre-processed all corpora using SpaCy for tokenization and sentence boundary detection. For the sentence similarity analysis, they subsampled the contract corpora (specifically NDAs) and the baseline corpora to a manageable size (200 documents each) due to computational intensity.
Key Insights and Ideas
- Contracts are Linguistically Constrained: The analysis strongly suggests that contract language, while natural, operates within a more restricted linguistic space than general English.
- Hapax Legomena as a Key Indicator: The significantly lower frequency of hapax legomena in contracts (e.g., 1/5th the frequency compared to baseline corpora) is a strong indicator of reduced vocabulary diversity.
- Sentence Similarity is Dramatically Higher: The most striking finding is the significantly higher sentence-level similarity in contract corpora, suggesting that entire phrases and sentence structures are reused more frequently.
- Implications for NLP: The reduced complexity (fewer rare words, higher sentence similarity) makes contract language potentially more amenable to certain NLP tasks, such as coreference resolution, by simplifying ambiguity.
- Repetition is Functional, Not Just Redundant: The paper argues that this repetition is a functional aspect of contract language, serving to create arrangements between parties rather than simply conveying new information, distinguishing it from genres like newswire.
Key Results
- Vocabulary Diversity: Contract corpora have substantially smaller vocabulary sizes compared to similarly sized non-contract corpora.
- Hapax Legomena: Contracts contain approximately 1/5th as many hapax legomena as baseline corpora. For example, the ratio of hapax legomena to tokens in Prime Contracts is about 1/120, compared to 1/35 for the Brown Corpus.
- Sentence Similarity: Sentences in contract corpora are, on average, about 20% more similar to one another than in baseline corpora. For NDAs, the average sentence similarity was found to be 20% higher.
- Near-Exact Matches: The number of sentences with very high similarity (scores between 0.98 and 1.0) increased dramatically in the NDA corpus as more documents were added, reaching over 12% at 200 documents, compared to less than 0.3% for the baseline corpora.
- Pronoun Usage: Pronouns like "she," "he," and "they" appear less often in contracts, with "it" being used more frequently, and parties often being referred to by name or by "we" and "you."
Limitations and Future Work
- OCR Errors: Despite filtering, some Optical Character Recognition (OCR) errors remain in the contract data, which could slightly affect the results.
- Subsampling for Sentence Analysis: Due to computational constraints, the sentence similarity analysis was performed on a subsample of 200 documents per corpus, which might not capture the full variability of larger corpora.
- Future Work: The authors suggest extending this analysis to other legal corpora (e.g., case reports) and exploring the classification and examination of speech acts within these corpora. They also note that further improvements in NLP preprocessing (like lemmatization and more robust OCR error removal) could amplify the observed differences.
Summary updated