TL;DR
This paper details the "Bank of English" project, a massive undertaking to morphologically and syntactically annotate 200 million words of English text. The project leverages and refines the ENGTWOL lexical analyzer and the ENGCG constraint grammar parser to create a rich, annotated corpus for linguistic research.
What Problem Does This Paper Address?
Creating large-scale, richly annotated linguistic corpora is a fundamental challenge in natural language processing and computational linguistics. Such corpora are crucial for training and evaluating NLP models, as well as for in-depth linguistic analysis. The "Bank of English" project addresses the need for a comprehensive, syntactically and morphologically annotated corpus of contemporary English, encompassing both written and spoken varieties. The challenge lies in efficiently and accurately processing such a vast amount of text, requiring robust and scalable analysis tools.
Methods
The project employs a multi-stage analysis pipeline:
- Preprocessing: Standardizes raw text and tokenizes it into a format suitable for subsequent analysis. This stage also handles various markup codes systematically and aims to document any corrections made to the text.
- ENGTWOL Lexical Analysis: A morphological analyzer that identifies word forms and their potential readings.
- ENGTWOL Lexicon Updating: A crucial step where words not found in the existing lexicon are identified and semi-automatically added. This involves classifying words by part-of-speech and generating new entries, often relying on external dictionaries and corpus concordances for clarification.
- ENGCG Morphological Disambiguation and Syntax: The English Constraint Grammar (ENGCG) parser is used for both morphological disambiguation (assigning the correct part-of-speech and inflectional tags) and syntactic analysis. ENGCG is a rule-based system that uses constraints to prune ambiguous analyses and determine syntactic relationships. The paper highlights the development and refinement of ENGCG's syntactic component, including mapping rules and syntactic constraints.
Datasets
- The Bank of English Corpus: The primary dataset, comprising approximately 200 million words of both written and spoken English. The paper details the breakdown of this corpus into 11 batches, each around 10 million words, supplied monthly by Harper-Collins Publishers.
- Collins COBUILD Dictionary (1987) and Collins English Dictionary (1991): Used as primary resources for lexicon updating.
- Test Corpus: A manually disambiguated syntactic test corpus of approximately 30,000 words, consisting of 16 individual text samples chosen to represent text type variation.
Key Insights and Ideas
- Scalability of Linguistic Analysis: The project demonstrates that it's feasible to apply sophisticated morphological and syntactic analysis to extremely large text corpora using automated systems like ENGTWOL and ENGCG.
- Importance of Lexicon Maintenance: Continuous updating of the lexicon is vital for achieving high coverage and accurate analysis, especially for non-standard or domain-specific vocabulary. The paper emphasizes the efficiency gains achieved through experience and increased lexicon coverage.
- Robustness of Constraint Grammar: ENGCG proves to be a fast and accurate rule-based system capable of parsing unrestricted text, with a mature morphological disambiguation component and ongoing development of its syntactic capabilities.
- Balancing Ambiguity and Accuracy: The paper discusses the trade-off between using heuristic rules to reduce ambiguity and the potential for introducing errors. The goal is to retain genuine ambiguity while pruning contextually inappropriate analyses.
- Documentation of Corrections: The practice of marking corrections made during preprocessing is a valuable aspect for ensuring transparency and reproducibility of the annotation process.
Key Results
- Corpus Size: The project aims to annotate a total of 200 million words.
- Processing Speed: The ENGCG system achieves a speed of approximately 400 words per second on a SUN SpareStation 10/30 for both morphological and syntactic annotation.
- Lexicon Coverage: The ENGTWOL lexicon, after updating, achieves a coverage of 97%-98% of all word-form tokens in running text.
- Error Rates: The current error rate for syntactic analysis of unrestricted running text is approximately 2%.
- Ambiguity Rate: The ambiguity rate for syntactic analysis is around 16.4% in a 0.5 million word sample, with prepositional attachment being a significant contributor.
- Lexicon Size: The ENGTWOL lexicon used in the Bank analyses contains approximately 75,000 entries.
Limitations and Future Work
- Ambiguity: While efforts are made to reduce ambiguity, a significant portion remains, particularly in prepositional attachment. More heuristic rules are suggested as a way to further prune these ambiguities.
- Test Corpus Size: The current syntactic test corpus (30,000 words) is considered sufficient for testing reliable rules but might need to be larger for rating the acceptability of heuristic rules.
- Manual Effort: Preparing large amounts of corrected and disambiguated data for testing is time-consuming.
- Future Work: The paper mentions that writing and documenting ENGCG syntax will be a primary focus in the coming months. A potential future step is to re-analyze the entire 200-million corpus near the end of the project to incorporate all improvements and ensure maximal uniformity and accuracy.
Summary updated