A “not-so-shallow” parser for collocational analysis

R. Basili, M.t. Pazienza, P. VelardiCOLING 1994

TL;DR

This paper introduces a "not-so-shallow" parsing strategy that significantly improves the accuracy of detecting word collocations, a crucial task for lexical acquisition. By incorporating more syntactic knowledge than typical shallow parsers, their approach achieves higher precision and recall while remaining computationally feasible for large-scale corpus analysis.

What Problem Does This Paper Address?

Collocational analysis, the study of words that frequently co-occur, is vital for understanding lexical acquisition and building richer NLP systems. However, existing methods often fall into two extremes: purely statistical approaches that require extensive human training data, or shallow parsers that produce noisy, unreliable results. This leaves a gap for automated systems that need high-quality lexical data without prohibitive human cost or computational overhead. The authors argue that current shallow techniques, while aiming for high coverage, often fail to capture important lexical relationships due to their limited syntactic understanding, leading to unacceptable noise for fully automated applications.

Methods

The core of this paper is the SSA (Syntactic Analyzer), a "not-so-shallow" parsing technique. Unlike traditional shallow parsers, SSA employs a discontinuous grammar approach, allowing it to detect not just adjacent word relationships but also longer-distance syntactic links. This is achieved through:

  • Discontinuous Grammar: A framework that allows for flexible parsing and the detection of dependencies between non-adjacent words, crucial for handling complex sentence structures like nested clauses and coordinate constructions.
  • Elementary Syntactic Links (ESLs): The analyzer identifies various types of binary and ternary syntactic relations (e.g., Noun-Verb, Verb-Noun, Noun-Preposition-Noun), providing a richer representation than simple SVO triples.
  • Morphological Lexicon: The system relies on a comprehensive morphological lexicon, which is more portable and requires less domain-specific training than statistical methods.
  • Post-Morphological Processing: An optional module to handle compound expressions and numbers, further refining the analysis.
  • Heuristic Rules (TEST_ON): Ad-hoc rules are integrated into the parsing process to avoid overgeneration and noise by checking for specific syntactic constraints (e.g., ensuring a skipped string doesn't contain a verb).

The key innovation is the integration of more syntactic knowledge, specifically through the use of skip rules and heuristic checks, to improve the reliability of detected collocations without sacrificing computational efficiency.

Datasets

The paper mentions using corpora from different linguistic styles and domains for evaluation:

  • Commercial Activities Corpus (CD): In telegraphic style, in Italian.
  • Legal Domain (LD): On taxation norms and laws, in Italian.
  • Remote Sensing (RSD) abstracts: In English.

For evaluation, they used a test set of 150 Italian legal sentences for which a full attribute grammar was already established as correct. For English, a test set of 10 remote sensing abstracts (approx. 1400 words, 67 sentences) was used, with evaluation performed manually due to the lack of a full English grammar.

Key Insights and Ideas

  • "Not-so-shallow" is the sweet spot: The paper convincingly argues that a moderate increase in syntactic sophistication, beyond purely shallow methods, can dramatically improve the quality of collocational data without incurring the full cost of deep parsing.
  • Syntax matters for collocation quality: Simply looking at word co-occurrence windows or basic partial parsing is insufficient. Understanding syntactic relationships is key to filtering out noise and identifying meaningful lexical associations.
  • Portability is achievable: The grammar-based approach, particularly with its reliance on a morphological lexicon rather than extensive statistical training, demonstrates better portability across different domains and languages compared to many statistical methods.
  • Heuristics are powerful noise filters: The TEST_ON rules, though simple, are crucial for preventing overgeneration and ensuring that detected links are syntactically plausible, even if they don't capture every single possible relation.

Key Results

  • Improved Recall and Precision: The SSA parser demonstrated significantly better performance compared to existing partial parsers. For example, for the N_P_N ESL type, recall was 69.1% and precision was 81.8% on the Italian legal corpus. For the English corpus, recall was generally high (e.g., 94% for N_P_N), though precision was lower (e.g., 54% for N_P_N) due to the lack of post-morphology.
  • Computational Efficiency: The SSA parser processed a test set of 150 Italian sentences in just 10 minutes on a Sun Sparc station, a stark contrast to the 6 hours required by a full grammar-based parser for the same task. The authors estimate the complexity of their rules to be O(n²), which is manageable for large corpora.
  • Reduced Ambiguity: Post-morphological processing reduced the initial ambiguity of words from 631 interpretations to 562 for a test set, showing an efficacy of about 50% reduction in initial ambiguity.

Limitations and Future Work

  • Recall for Complex Structures: The paper acknowledges that certain complex syntactic structures, like deeply nested relatives and inversions, can lead to lower recall for specific ESL types (e.g., N_V groups).
  • English Post-Morphology: The English application was less developed, lacking post-morphological processing, which contributed to lower precision for some ESL types (e.g., N_V pairs were affected by compounds).
  • "Fairness" of Evaluation: The authors discuss the challenges in evaluating performance, particularly the difficulty in defining a "fair" test material and the subjective nature of human judgment. They emphasize evaluating syntactic parsers on syntactic grounds, not semantic ones.
  • Future Work: The paper suggests that further improvements could be made by developing more sophisticated post-syntactic processing to handle phenomena like coordination and nested structures more effectively. The authors also express a continuous search for the "perfect balance" between syntax and statistical methods for optimal lexical acquisition.

Summary updated

BibTeX
@inproceedings{basili1994not,
  author = {R. Basili and M.T. Pazienza and P. Velardi},
  title = {{A “not-so-shallow” parser for collocational analysis}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1994},
}
Figure