Automatic Extraction of Subcorpora based on Subcategorization Frames from a Part-ofSpeech Tagged Corpus

Susanne Gahl

TL;DR

This paper introduces a novel tool for automatically extracting specific subcorpora from a large, part-of-speech tagged corpus. This method allows researchers to efficiently gather evidence for the subcategorization patterns of verbs, nouns, and adjectives, providing crucial data for linguistic analysis and computational linguistics applications.

What Problem Does This Paper Address?

Understanding how words combine syntactically (subcategorization) is fundamental to linguistics and natural language processing. While dictionaries and existing databases offer some information, they often lack direct corpus evidence or detailed frequency data for these patterns. This makes it difficult for researchers to verify dictionary claims, study the actual usage of words, and for computational models to learn realistic linguistic behavior. The paper addresses the need for a tool that can bridge this gap by automatically extracting relevant corpus data based on subcategorization frames.

Methods

The core of the paper's contribution is a system built using the Corpus Query Processor (CQP) and a custom macroprocessor. The process involves:

  1. Lemma-Subcorpus Creation: Initially, a subcorpus containing all occurrences of a target lemma is extracted.
  2. Syntactic Pattern Extraction: The system uses CQP queries, which act like a "chunk parser" using regular expressions over part-of-speech tags, lemmas, and morphosyntactic information. These queries are designed to identify specific syntactic structures (e.g., verb + noun phrase + prepositional phrase).
  3. Macroprocessor for User Specification: A key innovation is the macroprocessor, which allows users to define which subcorpora they want to create for a given lemma through a simple input file. This automates the generation of numerous specific subcorpora based on user-defined subcategorization frames.
  4. Cascading Queries: The system employs a cascading query strategy where results from earlier queries are removed from the corpus before subsequent queries are run. This prevents overcounting and ensures that more specific patterns are captured before more general ones.

Datasets

The primary dataset used is the British National Corpus (BNC), a 100-million-word corpus that is part-of-speech tagged. The authors also reference the COMLEX syntactic dictionary as a source of information for defining subcategorization frames.

Key Insights and Ideas

  • Bridging the Gap: The tool directly addresses the lack of corpus evidence and frequency data for subcategorization patterns, which is critical for both linguistic research and NLP model training.
  • Efficiency through Automation: The macroprocessor significantly streamlines the process of extracting specific linguistic data, which would otherwise be a laborious manual task.
  • Leveraging Existing Tools: The system effectively builds upon powerful existing tools like CQP, demonstrating how they can be combined for novel linguistic analysis.
  • Importance of Frequency: The paper highlights that statistical NLP and psycholinguistics increasingly rely on frequency information, making the ability to extract frequency data for specific syntactic patterns highly valuable.

Key Results

The paper presents a method for extracting subcorpora that document different subcategorization frames for verbs, nouns, and adjectives. The resulting subcorpora can be used to:

  • Provide evidence for the subcategorization properties of a given lemma.
  • Facilitate the selection of corpus lines for lexicographic research.
  • Determine the frequencies of different syntactic contexts for each lemma.

The paper provides examples of queries for various verb complement types (e.g., np, np_np, np_pp, np_vping) and illustrates how the system can handle complex noun phrase structures.

Limitations and Future Work

The authors acknowledge several limitations:

  • Reliance on Input: The tool relies on pre-defined subcategorization information (e.g., from dictionaries). It cannot automatically learn new frames that are missing from the input.
  • Unparsed Corpus: Working with an unparsed corpus means that certain "null" or "empty" constituents are not recognized, and ambiguities in prepositional phrase attachment can lead to errors.
  • Tag Accuracy: The accuracy of the extracted data is dependent on the quality of the part-of-speech tags and other annotations in the corpus.

Future work suggested includes:

  • Expanding searches to include predicative (vs. attributive) uses for adjectives and nouns.
  • Covering the entire set of complementation patterns described in the COMLEX syntactic dictionary for verbs.
  • The system's ability to identify passive structures, tough-movement, and common left-isolation constructions is noted as a current capability.

Summary updated

BibTeX
@inproceedings{gahl1998automatic,
  author = {Susanne Gahl},
  title = {{Automatic Extraction of Subcorpora based on Subcategorization Frames from a Part-ofSpeech Tagged Corpus}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1998},
}
Figure