TL;DR
This paper introduces a semi-automatic method to tailor broad-coverage grammars to specific sublanguages (like medical reports or equipment failure logs). By parsing sample texts and analyzing the grammar rules used, they create "trimmed" grammars that significantly speed up natural language processing and reduce parsing errors for those specific domains.
What Problem Does This Paper Address?
Natural language processing (NLP) systems often struggle with "sublanguages" – specialized forms of language used by particular communities (e.g., doctors, engineers). These sublanguages differ from standard language in their syntax, semantics, and discourse patterns. While general-purpose grammars can handle many constructs, they are often too broad, leading to inefficient processing and incorrect parses when applied to specialized domains. Manually creating a grammar for each sublanguage is a tedious and time-consuming process. This paper addresses the challenge of efficiently adapting NLP grammars to these specialized sublanguages without extensive manual effort.
Methods
The authors propose a semi-automatic procedure to identify sublanguage syntactic usage. The core of their method involves:
- Parsing with a Broad-Coverage Grammar: A sample corpus of text from the target sublanguage is parsed using a comprehensive, existing English grammar (the Linguistic String Grammar, augmented for fragments).
- Manual Review of Parses: The generated parse trees are manually reviewed to eliminate incorrect parses. This step is crucial for ensuring the quality of the data used for grammar trimming.
- Counting Production Usage: A program then counts how many times each production (a rule in the context-free grammar component) was used to build the correct parse trees for the entire corpus.
- "Trimming" the Grammar: This analysis results in a "trimmed" grammar, which consists only of the productions that were actually used in the sublanguage sample. The process also yields frequency information for these productions.
This approach automates the identification of domain-specific syntactic patterns, allowing for the creation of more efficient and accurate grammars for sublanguages.
Datasets
The authors applied their procedure to three text samples:
- Two sets of medical documents:
- A set of six patient documents (including history, examination, and plan of treatment). This sample yielded 236 correctly parsed sentences/fragments.
- A set of five hospital "discharge summaries" (similar to the first set, including history, examinations, and treatment summaries). This sample yielded 310 correctly parsed sentences/fragments.
- A set of equipment failure messages: Specifically, electrical equipment failure reports called "CASREPS" (a class of operational reports used by the U.S. Navy). This sample yielded 123 correctly parsed sentences/fragments.
These datasets were used to generate trimmed grammars and evaluate the effectiveness of the procedure.
Key Insights and Ideas
- Sublanguages are Significantly Simpler: Trimmed grammars for sublanguages are substantially smaller than the full English grammar, reflecting a more restricted range of syntactic constructions and modifiers.
- Overlap in Sublanguage Structures: Despite differences, the three sublanguage grammars examined showed significant overlap in their syntactic constructs, suggesting common patterns across related domains.
- Frequency as a Signal: The frequency of specific syntactic constructions varies significantly across sublanguages, providing valuable domain-specific information. For example, noun phrases modifying other nouns were much more frequent in equipment failure reports than in medical documents.
- Practical Benefits of Trimming: Using a trimmed grammar leads to noticeable speedups in parsing, especially for complex sentences that require extensive backtracking with a broad-coverage grammar. It also helps reduce spurious, incorrect parses.
Key Results
- The trimmed grammars produced were significantly smaller than the broad-coverage grammar. For instance, the "medical records" grammar used by the NYU Linguistic String Project had 255 non-terminal symbols and 744 productions, while the trimmed grammars for the sublanguages had between 124-134 non-terminals and 220-282 productions.
- The number of options for sentence objects, sentential modifiers, and right modifiers of nouns were substantially restricted in the sublanguage grammars compared to the full English grammar.
- Parsing speedup was observed, with gains of about 25% for simple sentences and approximately a factor of 3 for long, complex sentences requiring extensive backtracking.
- The frequency of incorrect parses decreased slightly (<3%) when using the trimmed grammar.
Limitations and Future Work
- Partial Discovery: The authors acknowledge that their procedure is "partial" because it cannot identify entirely new extensions to the base grammar. It also relies on manual review, making it "semi-automatic."
- Future Work:
- Parse Selection: They plan to use the gathered frequency information to help select the correct parse when multiple parses satisfy both syntactic and sublanguage semantic constraints.
- Selectional Patterns: They are extending this work to develop a discovery procedure for sublanguage selectional (semantic) patterns by analyzing co-occurrence statistics of semantic classes.
Summary updated