Computing Logical Form on Regulatory Texts

Nikhil Dinesh, Aravind K. Joshi, Insup LeeEMNLP 2011

Introduction

Presented at EMNLP 2011, this research addresses the critical challenge of translating dense, ambiguous government regulations into machine-readable formats. Automated regulatory compliance requires converting legal prose into formal logic, a task complicated by the inherent linguistic ambiguities of legal text.

The Challenge of Scope Ambiguity

A primary bottleneck in formalizing regulations is scope ambiguity. In a sentence like "Every kid climbed a tree," it is unclear if there is one specific tree or many. In legal contexts, these ambiguities extend beyond simple quantifiers to include modals ("shall," "may"), negations, and complex conditional phrases. While previous NLP research often focused on simple noun phrases, this work tackles the full complexity of regulatory requirements found in documents like the FDA Code of Federal Regulations.

From Parse Trees to Abstract Syntax Trees

The authors propose a modular approach using Abstract Syntax Trees (ASTs) as an intermediate representation. Unlike standard parse trees, ASTs explicitly encode the scopal ordering of operators.

The process begins with Processed Parse Trees (PPTs), where operators and their restrictors are identified but not yet ordered. The system then resolves the relative scope to produce the final AST.

An Abstract Syntax Tree (AST) representing the resolved scope for a regulatory requirement.

This representation allows the system to bypass the complexities of full universal semantic parsing while providing a structure that is "logic-ready" for downstream conformance checking.

Machine Learning for Scope Resolution

To determine the correct scopal ordering, the authors frame the task as a supervised ranking problem. Using a Maximum Entropy model, they predict the relationship between pairs of operators ($o_i$ and $o_j$) to determine if $o_i$ "outscopes" $o_j$.

The model leverages several key features:

  • Operator Type: The specific subtype of a determiner or modifier.
  • Syntactic Position: The operator's location relative to the main verb.
  • Inverse Linking Heuristics: Specialized logic to handle cases where nested operators (e.g., "any" in "samples of any lot") take wide scope over their containing phrases.

Key Insights: The Role of Determiner Types

A major finding of the study is the importance of operator "type" in determining scope. In legal text, a critical distinction exists between de re scope (preconditions describing existing objects) and de dicto scope (postconditions describing actions to be performed).

Distribution of De Re scope across different determiner types in the FDA corpus.

As shown in the data, certain determiners are highly predictive: universal determiners in this corpus always took de re scope, while existential ones never did. Modals like "shall" were found to almost always signal de dicto scope.

Results and Performance

The system was evaluated on a specialized corpus of 195 sentences from the FDA Code of Federal Regulations. Despite the complexity of these sentences (averaging 30 words each), the model achieved significant results:

  • Pairwise Ordering: A 90.6% F-score in predicting the relative scope of operator pairs.
  • Full AST Accuracy: The model correctly predicted the entire scope tree for 36.2% of sentences—a nearly nine-fold improvement over the surface-order baseline of 4.2%.

Conclusion and Future Work

This work demonstrates that supervised ranking with rich linguistic features can effectively resolve scope ambiguity in complex regulatory texts. Future improvements will focus on automating the creation of Processed Parse Trees and addressing discourse-level challenges like anaphora and cross-references within legal documents.

Summary updated

BibTeX
@inproceedings{dinesh2011computing,
  author = {Nikhil Dinesh and Aravind K. Joshi and Insup Lee},
  title = {{Computing Logical Form on Regulatory Texts}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2011},
}
Figure