The RASP (Robust Accurate Statistical Parsing) system is a modular toolkit designed to transform raw, unannotated text into structured syntactic representations. RASP 2.0 represents a significant upgrade to the original toolkit, featuring an expanded grammar, improved part-of-speech (PoS) tagging, and a semantically-grounded output format. By combining traditional unification-based rules with modern semi-supervised training techniques, the system achieves a 76.3% F1 score on the DepBank benchmark, providing a robust pipeline for downstream tasks like summarization and information extraction.
System Architecture
The RASP system operates as a pipeline of Unix-style filters, processing text through several specialized modules to move from raw input to a ranked set of parses.
- Tokenization & Sentence Splitting: Deterministic finite-state rules handle raw text, including complex cases like quotations and XML markup.
- PoS Tagging: A first-order Hidden Markov Model (HMM) uses the 150-tag CLAWS set, enhanced with a semi-automatically generated lexicon to better handle rare or unseen words.
- Morphological Analysis: A deterministic transducer reduces words to their lemmas and inflectional affixes (e.g., "running" $\rightarrow$ "run" + "-ing").
- The Parser: A non-deterministic LALR(1) parser utilizes a wide-coverage grammar of 689 unification-based rules, expanded from the original 400.
- Parse Ranking: A statistical model ranks valid parses using probabilities computed via bootstrapping on unlabelled data, which facilitates easier retraining for new domains.
The Grammatical Relation Hierarchy
A core innovation in RASP 2.0 is the introduction of Grammatical Relations (GRs) organized in a subsumption hierarchy. Unlike flat dependency lists, this structure allows for more nuanced evaluation and semantic inference.
By organizing relations such that a "non-clausal subject" (ncsubj) is a subtype of "subject" (subj), which in turn is an "argument" (arg), the system supports "soft" evaluation. If a specific subtype is missed but the general relationship is correctly identified, the output remains valuable for downstream applications. This predicate-argument structure is more stable across different grammar theories than traditional tree branches, making RASP easier to compare with other parsing systems.
Performance and Evaluation
The system was evaluated using the DepBank benchmark, a re-annotated version of the PARC dependency bank consisting of 560 sentences from the Wall Street Journal. Key results include:
- Overall Accuracy: A microaveraged F1 score of 76.3% across all relations.
- PoS Tagging: Accuracy exceeded 97% on WSJ text.
- Morphological Precision: The analyzer achieved an error rate of less than 0.07%.
- Grammar Coverage: The parser found a full sentence parse for 85% of the Susanne corpus and 84% of the WSJ test data.
In cases where a full parse cannot be found (approximately 15-16% of sentences), RASP employs "partial parsing" as a safety net, returning a sequence of the most probable partial parses to ensure system robustness.
Conclusion
RASP 2.0 provides a comprehensive and flexible framework for syntactic analysis. While it demonstrates high accuracy and coverage, future work aims to improve domain adaptation and refine the grammar to address the remaining parsing failures. The system's modularity and hierarchical output make it a powerful tool for complex NLP tasks requiring deep syntactic understanding.
Summary updated