Introduction
Extracting timelines from Electronic Medical Records (EMRs) is essential for longitudinal patient analysis, including disease progression and treatment efficacy. This task involves identifying medical events, time expressions, and their temporal relationships (e.g., "Event A contains Event B"). While BERT-based models have set new benchmarks in this field, they typically rely on a computationally expensive multi-pass approach, processing the same sentence for every candidate entity pair. This paper introduces a "one-pass" architecture that extracts all relations in a single forward pass, significantly improving efficiency without compromising accuracy.
Methodology
The proposed model shifts from standard relation classification to a comprehensive relation extraction framework.
One-Pass Encoding and Global Context
Unlike traditional models that re-encode sequences for each pair, this architecture encodes the input sequence once. To handle long-distance relations common in clinical narratives, the model utilizes an entity-aware self-attention mechanism. A critical innovation is the inclusion of a global embedding ($G$), created by average-pooling the entire sequence. This global context is concatenated with local entity embeddings ($e1, e2$) to facilitate broader inference.
Multi-Task Learning (MTL)
The model is trained jointly on two distinct but related tasks:
- CONTAINS: Determining the temporal containment relationship between two entities.
- DocTimeRel: Classifying the temporal status of an entity relative to the Document Creation Time (e.g., BEFORE, AFTER, OVERLAP).
This multi-task setup serves as a regularizer, enhancing the model's ability to generalize across different medical domains.
Experimental Setup
The model was evaluated using the THYME corpus, which contains annotated clinical text. The Colon Cancer subset was used for primary training and development, while the Brain Cancer subset served as a holdout set to test cross-domain generalizability.
Results and Discussion
The one-pass model demonstrates that "Green AI" can achieve state-of-the-art results while being environmentally and computationally responsible.
Efficiency Gains
The one-pass approach reduces the number of training instances from approximately 497,000 to just 20,000. This results in a 25-fold increase in efficiency compared to multi-pass baselines, drastically reducing GPU compute requirements.
Performance and Generalization
On the Colon Cancer test set, the model achieved an F1 score of 0.686 on the CONTAINS task, matching the performance of multi-pass models that required additional silver data. Furthermore, on the unseen Brain Cancer domain, the MTL model outperformed previous SOTA models with an F1 of 0.582, proving its robustness.
Conclusion
This research proves that architectural optimizations can eliminate the need for redundant computation in clinical NLP. By combining one-pass encoding with global context and multi-task learning, the authors provide a lightweight yet powerful framework suitable for deployment with various domain-specific pre-trained language models.
Summary updated