An end-to-end neural model based on cliques and scopes for frame extraction in long breast radiology reports

Perceval Wajsburt, Xavier TannierACL 2023

Sure, here's a blog-style summary of the paper:

TL;DR

This paper introduces a novel end-to-end neural model for extracting structured information (frames) from long, complex breast radiology reports. Their approach tackles the challenge of overlapping and multi-sentence entities by leveraging a clique-based method and a new concept called "scope relations" to better model contextual information.

What Problem Does This Paper Address?

Extracting structured information from clinical text, like radiology reports, is crucial for tasks such as automated summarization, clinical decision support, and data mining. However, these reports are often lengthy and contain complex, overlapping entities (frames) that span multiple sentences or even paragraphs. Existing methods struggle with these challenges, particularly when frames share mentions or triggers. This paper aims to address this by developing a model that can effectively handle these overlapping and discontinuous entities in long breast radiology reports.

Methods

The authors propose an end-to-end neural model that operates in a pipeline:

  1. Mention Extraction (NER): Identifies potential entities (mentions) within the text using a BIOUL tag scheme and parallel CRF layers, allowing for overlapping entities.
  2. Mention Normalization: Classifies or normalizes extracted mentions to specific values (e.g., "left" or "right" for laterality).
  3. Frame Extraction: This is the core innovation. Instead of directly predicting frame memberships, they use a clique-based approach. They first compute pairwise relation scores between mentions using:
  4. Biaffine Relation Scores: A baseline approach using attention mechanisms that incorporate relative distances between mentions.
  5. Scope Relation Scores: A novel concept where "scopes" represent contiguous text zones to which a mention applies its meaning. This helps model contextual information and relationships between mentions more effectively. They then extract maximal cliques of mentions that agree on belonging to the same frame.

  6. Frame Classification: Predicts values for fields within a frame for which no explicit mention was found (e.g., temporality).

The model also incorporates knowledge injection through data augmentation (splitting documents, creating synthetic sentences) and output constraints (e.g., "left" and "right" are exclusive).

Datasets

The authors introduce a new annotated corpus of 120 French breast imaging clinical reports. This corpus is used for training (80 documents) and evaluation (40 documents). The reports contain five types of objects: ACR scores, breast density scores, diagnostic procedures, therapeutic procedures, and lesions.

Key Insights and Ideas

  • Scope Relations are Powerful: The concept of "scopes" significantly improves performance by allowing the model to reason about text spans and contextual information, which is crucial for understanding relationships in long documents.
  • Clique-Based Frame Extraction: Moving beyond simple pairwise relations, extracting maximal cliques of mentions helps to group related entities into frames more robustly, especially in the presence of overlaps.
  • End-to-End Approach: The model's end-to-end nature simplifies the pipeline and allows for joint training of all components, potentially leading to better overall performance.
  • Data Efficiency: The model shows promising performance even with a small amount of annotated data, suggesting its utility for pre-annotating new domains and reducing manual annotation effort.
  • Knowledge Injection Matters: Both data augmentation and output constraints contribute to improved performance, highlighting the benefit of incorporating domain knowledge.

Key Results

  • The proposed model achieves strong performance on frame extraction, with an F1 score of 85.3 for Frame Support and 72.2 for Frame Label on the test set.
  • Ablation studies show that removing scopes leads to a significant performance drop (5.3 pts for Frame Label F1).
  • The model demonstrates good data efficiency, achieving reasonable performance with as few as 10-20 annotated documents.
  • Data augmentation techniques, particularly synthetic lexicon sentences, provide a notable boost in performance, especially for the Frame Label metric.

Limitations and Future Work

  • Ambiguity in Frame Splitting: The authors acknowledge that deciding when to split a single object into multiple frames can be ambiguous.
  • Frame-Coreference Prediction: The study does not address the prediction of frame-coreference (grouping frames into objects), which is a potential area for future work.
  • Generalizability: While promising, the model's performance on other structured entity extraction tasks or different clinical domains remains to be explored.
  • Future Work: The authors suggest evaluating their approach on other structured entity extraction tasks, such as event extraction.

Summary updated

BibTeX
@inproceedings{wajsburt2023end,
  author = {Perceval Wajsburt and Xavier Tannier},
  title = {{An end-to-end neural model based on cliques and scopes for frame extraction in long breast radiology reports}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure