Automatic extraction of subcorpora based on subcategorization frames from a part-of-speech tagged corpus

Susanne Gahl

TL;DR

This paper introduces a novel tool for automatically extracting specific subcorpora from a large, part-of-speech tagged corpus. This allows researchers to efficiently find and quantify evidence for how verbs, nouns, and adjectives combine with different syntactic complements, addressing a critical gap in linguistic research and lexicography.

What Problem Does This Paper Address?

Understanding how words combine syntactically (their subcategorization frames) is fundamental to linguistics and natural language processing. While dictionaries provide some of this information, they often list expected patterns rather than actual occurrences and lack frequency data. Existing corpus-based methods for inferring these frames can be cumbersome and don't easily provide evidence or frequency counts for specific patterns. This leaves a gap for researchers needing to verify dictionary entries, discover new patterns, or study the frequency of syntactic constructions, which is crucial for psycholinguistic research on sentence processing.

Methods

The authors developed a tool that leverages the Corpus Query Processor (CQP) and a custom macroprocessor to extract subcorpora based on subcategorization frames. The process involves:

  1. Lemma-Specific Subcorpora: First, a subcorpus containing all occurrences of a target lemma is created.
  2. Syntactic Pattern Extraction: The tool then uses a series of CQP queries, written in a regular grammar that simulates a chunk parser, to identify specific syntactic patterns. These queries are designed to be "positive" (looking for specific structures) and "negative" (excluding certain structures to avoid false positives).
  3. Cascading Queries: Queries are applied sequentially, with the results of earlier queries being removed from the pool of potential matches for later queries. This "pre-emption" strategy ensures that more specific patterns are identified before more general ones (e.g., passive constructions are identified before general verb-prepositional phrase complements).
  4. Macroprocessor for User Specification: A macroprocessor allows users to define which subcorpora to create for a given lemma by specifying desired complement types (e.g., noun phrases, prepositional phrases, infinitives) and their order relative to the target lemma.

Datasets

The primary dataset used is the 100 million word British National Corpus (BNC), which is part of-of-speech tagged. The tool is also used within the FrameNet project, an NSF-funded initiative to create a lexical database based on Frame Semantics.

Key Insights and Ideas

  • Bridging the Gap: The tool effectively bridges the gap between theoretical linguistic knowledge (dictionaries) and empirical evidence from large corpora, providing concrete examples and frequency data.
  • Efficiency for Lexicography and Research: By automating the extraction of specific subcorpora, the tool significantly speeds up the process of gathering evidence for subcategorization properties, which is invaluable for lexicographers and researchers.
  • Frequency as Key Evidence: The paper emphasizes the importance of frequency information for understanding syntactic patterns, particularly for psycholinguistic research on sentence processing.
  • Handling Ambiguity: The cascading query strategy is a clever way to manage ambiguity and ensure that more specific syntactic structures are identified correctly, even in an unparsed corpus.

Key Results

The paper presents a method and a tool that can extract subcorpora documenting different subcategorization frames for verbs, nouns, and adjectives. The resulting subcorpora can be used to:

  • Provide evidence for the subcategorization properties of a given lemma.
  • Facilitate the selection of corpus lines for lexicographic research.
  • Determine the frequencies of different syntactic contexts for each lemma.

The paper includes examples of queries for various verb complement types (e.g., np_np, np_pp, np_vpto) and demonstrates how the tool can identify complex structures like coordinated passives.

Limitations and Future Work

  • Reliance on Input: The tool relies on pre-existing subcategorization information (e.g., from dictionaries or user input). It cannot automatically learn new subcategorization frames that are missing from its input.
  • Unparsed Corpus: Working with an unparsed corpus means that certain "null" or "empty" constituents might not be recognized, and ambiguities in prepositional phrase attachment can lead to errors.
  • Tag Accuracy: The accuracy of the search results is dependent on the accuracy of the part-of-speech tags and other annotations in the corpus.

Future work planned includes:

  • Searches for predicative versus attributive uses of adjectives and nouns.
  • Expansion of verb searches to cover the entire set of complementation patterns described in the COMLEX syntactic dictionary.

Summary updated

BibTeX
@inproceedings{gahl1998automatic,
  author = {Susanne Gahl},
  title = {{Automatic extraction of subcorpora based on subcategorization frames from a part-of-speech tagged corpus}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1998},
}

Loading related content...

Figure