TL;DR
This paper presents a computational analysis of predicational structures in a large corpus of American English, revealing significant syntactic differences between informative and imaginative prose. It demonstrates that sentence length is not a direct predictor of syntactic complexity, as defined by the number of predications per sentence.
What Problem Does This Paper Address?
Linguistic analysis often seeks to quantify and compare the complexity of language use across different genres or styles. While sentence length is an easily measurable metric, it's unclear how well it reflects deeper syntactic complexity. This paper tackles this by computationally analyzing "predications" – the core units of meaning within sentences – across various genres of written English to understand how syntactic complexity, as measured by predication density, varies and relates to more superficial features like sentence length.
Methods
The core of this research relies on a computational analysis of the "Brown Corpus," a one-million-word collection of present-day American English texts. The corpus was "tagged" with a detailed grammatical symbol system (82 categories), allowing for algorithmic retrieval of syntactic information. The authors developed a method to identify and count both finite (tensed verbs) and non-finite (infinitives, gerunds, participles) predications within sentences. This involved a sophisticated retrieval algorithm, essentially a finite-state automaton, designed to handle continuous and discontinuous verbal constructions, including truncated forms.
Datasets
The primary dataset used is the Brown Corpus, a one-million-word corpus of present-day American English. This corpus is divided into 15 genres, further categorized into "Informative Prose" (genres A-J) and "Imaginative Prose" (genres K-R). No new benchmarks were introduced; the paper analyzes the existing, grammatically annotated Brown Corpus.
Key Insights and Ideas
- Sentence Length vs. Syntactic Complexity: The most striking insight is that longer sentences don't necessarily mean more complex sentences in terms of predication density. This challenges the intuitive assumption that sentence length is a direct proxy for syntactic complexity.
- Genre-Specific Syntactic Styles: Informative prose consistently exhibits longer sentences and a higher number of words per predication compared to imaginative prose. This suggests that informative writing, on average, uses more elaborate nominal structures to convey information within a sentence.
- The Nature of Non-Finite Predications: Informative prose uses a larger proportion of non-finite predications (gerunds and participles) than imaginative prose. This contributes to a potentially denser informational structure within sentences.
- Implications for Parsing: The paper highlights the challenges in computational parsing, particularly with discontinuous verbal constructions and truncated forms, underscoring the need for robust algorithms that can handle these linguistic phenomena.
Key Results
- Sentence Length: Informative prose genres generally have higher mean sentence lengths (e.g., 24.23 words in "Miscellaneous" to 18.63 words in "Skills and hobbies") than imaginative prose genres (e.g., 17.64 words in "Humor" to 12.81 words in "Mystery and detective fiction").
- Predications per Sentence: While informative prose tends to have more predications per sentence on average, the difference is less pronounced than sentence length differences. The correlation is not direct.
- Words per Predication: Informative prose requires significantly more words per predication (average 7.54) than imaginative prose (average 5.69). This points to more complex nominal structures in informative texts.
- Finite vs. Non-Finite Predications: Informative prose has a higher percentage of non-finite predications (31.91%) compared to imaginative prose (24.04%).
Limitations and Future Work
- Definition of Complexity: The paper acknowledges that its definition of syntactic complexity is limited to the number of predications per sentence. Other aspects of complexity, such as the depth of embedding or the types of grammatical constructions used, are not explicitly measured.
- "Zero" Predications: Sentences without any explicit verb (e.g., "Just our luck!") were included in statistics, though they represent a small percentage (3.4%) of the corpus.
- Future Work: The authors state that ongoing research with the Brown Corpus aims to provide further insights into the syntactic structure of English texts and their stylistic properties, as well as to advance problems in automatic parsing. The paper itself is a segment of a larger parsing algorithm.
Summary updated