Parallelism in Coordination as an Instance of Syntactic Priming: Evidence from Corpus-based Modeling

Amit Dubey, Patrick Sturt, Frank KellerEMNLP 2005

TL;DR

This paper demonstrates that the "parallelism effect" – where we process coordinated structures faster if they have similar syntax – isn't just about coordination. Using corpus analysis, they show this structural repetition preference extends to arbitrary syntactic structures, within and between sentences, suggesting it's a general syntactic priming mechanism in language processing. This has implications for understanding how we produce and comprehend language, and potentially for improving natural language processing models.

What Problem Does This Paper Address?

Psycholinguistics has observed a "parallelism preference" where people process coordinated phrases (like "a long novel and a short poem") more quickly when the two parts have similar grammatical structures. This effect has been shown for various types of coordination. However, the underlying mechanism is debated. One hypothesis is that it's a specialized "copying" mechanism for coordination. An alternative, and the focus of this paper, is that it's a manifestation of a more general "syntactic priming" phenomenon, where people tend to repeat recently used syntactic structures. The problem this paper addresses is to determine if the parallelism effect is specific to coordination or if it's a broader instance of syntactic priming, by examining large-scale corpus data.

Methods

The authors employ a corpus-based approach using adaptation probabilities, a method originally developed for lexical priming. This involves dividing a corpus (or documents within it) into a "prime set" and a "target set." They then measure how frequently a specific syntactic construction appears in the target set, given whether or not it appeared in the prime set.

Specifically, they define:

  • Prior Probability: The baseline probability of a construction appearing.
  • Positive Adaptation Probability: The probability of a construction appearing in the target set, given it appeared in the prime set.

A higher positive adaptation probability than the prior suggests a priming effect – the construction is more likely to reappear if it was recently "primed." They apply this to various syntactic constructions (e.g., NP with a relative clause, NP with a PP modifier, simple NP) and analyze them at different granularities:

  1. Within-sentence parallelism: Examining NPs within the same sentence, but not necessarily coordinated.
  2. Between-sentence parallelism: Examining structures in adjacent sentences.
  3. Within-document parallelism: Examining structures across halves of entire documents.
  4. Inter-speaker parallelism (in dialog): Examining structures between different speakers in a conversation.

Datasets

The study utilizes portions of the Penn Treebank, specifically:

  • The Wall Street Journal (WSJ) corpus (written text).
  • The Brown corpus (written text).
  • The Switchboard corpus (spontaneous dialog).

These corpora are used to analyze written text and spoken dialog, respectively, allowing for a broad investigation of syntactic repetition across different communication modalities.

Key Insights and Ideas

  • Parallelism is Pervasive: The most significant finding is that the tendency to repeat syntactic structures (parallelism) is not confined to coordinated phrases. It extends to arbitrary NP structures, occurs within sentences, between sentences, and even across documents.
  • Syntactic Priming as the Mechanism: This broad applicability strongly suggests that the parallelism effect is a consequence of a general syntactic priming mechanism, rather than a specialized coordination-specific process.
  • Coordination Amplifies Priming: While parallelism exists broadly, it is strongest in coordinate structures. This might explain why previous experimental studies focused solely on coordination and found robust effects, while studies on other structures were less conclusive.
  • Production vs. Comprehension: The study provides evidence for parallelism in production data (corpora), complementing previous findings from comprehension studies. This suggests that the same underlying priming mechanisms might operate in both.
  • Implications for NLP: The findings suggest that natural language processing models, particularly probabilistic parsers that often assume sentence independence, could be improved by incorporating mechanisms that account for structural repetition and dependencies across sentences.

Key Results

  • Significant Adaptation: In all corpora and across all tested NP types, the positive adaptation probability was significantly higher than the prior probability for within-sentence, between-sentence, and within-document analyses. This indicates a robust structural repetition effect.
  • Strongest in Coordination: Figures comparing coordination (Experiment 1) with broader analyses (Experiment 2) show that the difference between prior and adaptation probabilities is markedly larger for coordinated structures.
  • Dialog Findings: In dialog, adaptation was found between speakers, supporting the idea of shared representations or interactive alignment. However, adaptation within a single speaker's turns was less pronounced, which the authors suggest might be due to decay effects or methodological limitations.

Limitations and Future Work

  • Smaller Effects Outside Coordination: The parallelism effect was generally smaller for non-coordinated structures compared to coordinated ones. The authors speculate this might be due to inherent constraints or expectations in coordination that amplify the effect.
  • Dialog Within-Speaker Analysis: The weaker within-speaker adaptation in dialog is noted as a limitation, with potential explanations including decay processes and the need for more elaborate experimental designs.
  • Future Work: The authors suggest further experiments to investigate why coordination shows stronger parallelism. They also plan to explore the application of these findings to probabilistic parsing, aiming to build models that better capture inter-sentence dependencies.

Summary updated

BibTeX
@inproceedings{dubey2005parallelism,
  author = {Amit Dubey and Patrick Sturt and Frank Keller},
  title = {{Parallelism in Coordination as an Instance of Syntactic Priming: Evidence from Corpus-based Modeling}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2005},
}

Loading related content...

Figure