TL;DR
This paper investigates how the way we learn language – whether by observing a single person's output (production) or a mix of many people's (perception) – affects the grammars we build. They found that learning from a single individual leads to richer, more complex grammars with more unique constructions, suggesting that much of linguistic diversity is hidden in individual usage.
What Problem Does This Paper Address?
Usage-based models of language acquisition posit that we learn language by encountering and generalizing from real-world language use. However, most computational models of grammar induction typically use large corpora that aggregate the language of many unknown individuals. This "perception-based" approach assumes that exposure to a diverse set of speakers leads to the discovery of general linguistic patterns. This paper questions whether this aggregation masks important individual differences. Specifically, it asks:
- Does the way we learn language (production vs. perception) influence the growth rate of our grammars and lexicons?
- Does it affect how similar the learned grammars are to each other?
- Does individual variation play a significant role in grammar induction, and if so, how does it manifest?
By contrasting models trained on single individuals' language with those trained on aggregated corpora, the authors aim to reveal the impact of individual linguistic idiosyncrasies on grammar formation.
Methods
The researchers employed a computational construction grammar (CxG) framework. In CxG, grammars are viewed as a set of symbolic units of varying complexity, from words to abstract patterns, all having both form and meaning. The core of their methodology involved:
- Grammar Induction Algorithm: An association-based beam search algorithm was used to identify sequences of "slot-constraints" that form grammatical constructions. This approach is designed to efficiently explore a large hypothesis space of potential grammars.
- Production vs. Perception Simulation:
- Production-based: Models were trained on corpora representing the language output of a single individual.
- Perception-based: Models were trained on corpora representing an aggregation of many different individuals' language use.
- Growth Curve Analysis: The growth of grammars and lexicons was measured by incrementally combining sub-corpora and observing the increase in learned representations (constructions and lexical items). This was quantified using the "a" parameter from Heap's Law, which describes power-law distributions in language.
- Grammar Similarity Measurement: The Jaccard distance was used to quantify the dissimilarity between grammars learned from different sub-corpora. A higher Jaccard distance indicates less overlap and greater difference between grammars.
- Register Variation: To ensure findings were not specific to a single type of language use, experiments were conducted across multiple registers of written English.
Datasets
The study utilized existing corpora and simulated data across six registers:
- Academic Articles: Used for both production (AC-IND) and perception (AC-AGG) conditions.
- Published Books (Project Gutenberg): Used for both production (PG-IND) and perception (PG-AGG) conditions.
- Background Registers (Perception-based only):
- Tweets (TW-AGG)
- Web Crawled data (CC-AGG)
- Wikipedia Articles (WI-AGG)
- News Articles (NW-AGG)
Each of these datasets was divided into 100k word increments, with up to 2 million words used for training in total. The "production" corpora were carefully curated to represent single authors (e.g., Dickens, Austen), while "perception" corpora aggregated language from many unknown speakers within the same register.
Key Insights and Ideas
- Individual Idiosyncrasy Matters: The most striking finding is that learning from a single individual's language (production) results in significantly more complex grammars with a higher number of constructions compared to learning from aggregated data (perception), given the same amount of exposure.
- Hidden Linguistic Diversity: This suggests that much of the unique linguistic variation within a language resides in individual speakers' grammars, and this diversity is largely obscured when only aggregated corpora are considered.
- Grammar vs. Lexicon Growth: While both grammars and lexicons grow with more data, grammars tend to grow more slowly and level off sooner than lexicons. This aligns with the idea that grammatical constructions are more abstract and complex than individual lexical items.
- Grammars Diverge, Lexicons Converge: Production-based grammars show greater pairwise distances (less similarity), indicating they diverge and retain individual characteristics. Perception-based grammars, conversely, show less variation and tend to converge on more stable, shared representations.
Key Results
- Steeper Grammar Growth Curves: Production-based models exhibited significantly steeper growth curves for grammar induction compared to perception-based models. For instance, in the published books register, the "a" parameter for grammar growth was 0.716 for production vs. 0.652 for perception.
- Larger Grammars: At 2 million words, production-based grammars contained more constructions (e.g., 25.7k for AC-IND vs. 16.2k for AC-AGG).
- Higher Grammar Dissimilarity: Violin plots and Jaccard distance calculations showed that grammars learned from individual productions were significantly more dissimilar to each other than grammars learned from aggregated data. The variance in Jaccard distances was notably higher for production-based conditions.
- Lexicon Growth Unaffected: The growth of the lexicon was largely unaffected by whether the data came from a single individual or an aggregation of many.
Limitations and Future Work
- Limited Registers: While multiple registers were used, the study focused on written English. Future work could explore spoken language and other languages to see if these findings generalize.
- Simulated Production: The "production" corpora were constructed from single authors. While this simulates individual output, it doesn't fully capture the dynamic nature of an individual producing language in real-time.
- Computational Constraints: The study acknowledges that finding 2 million words from 20 different individuals for the production condition was logistically challenging, leading to the use of 20 distinct authors. Future work could explore larger numbers of individuals.
- Future Directions: The authors suggest that their findings have implications for how we model language acquisition and understand linguistic variation. They highlight the need to account for individual differences in computational models and suggest that aggregated corpora might only reveal the "tip of the iceberg" of linguistic constructions. Further research could explore how these individual-specific constructions are shared or transmitted within communities.
Summary updated