Leveraging Language Models for Automated Patient Record Linkage

Mohammad Beheshti, Lovedeep Gondara, Iris ZacharyarXiv 2025 · OpenReview

TL;DR

This paper explores using large language models (LLMs) to automate the challenging task of patient record linkage in healthcare, a crucial step for integrating fragmented data. They found that fine-tuned LLMs show great promise for both blocking (reducing candidate pairs) and matching (identifying true matches), significantly improving efficiency over traditional methods.

What Problem Does This Paper Address?

Healthcare data is notoriously fragmented, scattered across various systems like hospitals, labs, and physician offices. This makes it difficult to get a complete picture of a patient's health journey. Patient record linkage, also known as entity resolution, is the process of connecting these disparate records to a single individual. Traditional methods, like deterministic or probabilistic linkage, can be time-consuming, require manual review, and struggle with data inconsistencies. This paper investigates whether modern LLMs can automate and improve this critical process.

Methods

The study breaks down record linkage into two main stages: blocking and matching.

  • Blocking: This is the initial step to reduce the number of record pairs that need to be compared. The authors fine-tuned a RoBERTa model to generate sentence embeddings for patient records. By comparing the semantic similarity of these embeddings, they could group similar records into "blocks," drastically reducing the search space.
  • Matching: Once candidate pairs are generated by blocking, the matching step determines if they refer to the same individual. They experimented with several LLMs in both "fine-tuned" (where the model is trained on specific data) and "zero-shot" (where the model performs the task with no or minimal task-specific training) settings. For fine-tuning, they used a classification head on top of RoBERTa. For generative LLMs like Mistral and Llama, they used a structured prompt template to guide the model's decision.

Datasets

The study utilized real-world healthcare data from the Missouri Cancer Registry and Research Center. Two distinct datasets were used:

  • Dataset A: Primary patient database of the cancer registry (CRS Plus).
  • Dataset B: ePath Reporting database (eMaRC Plus) containing pathology records.

Data from 2022 was used for fine-tuning the language models, while data from 2021 was used for testing. The ground truth labels for record pairs were created using a probabilistic linkage approach with the Match*Pro software, followed by manual review.

Key Insights and Ideas

  • LLMs are powerful for both blocking and matching: The study demonstrates that LLMs can effectively handle both stages of record linkage, offering a more automated and potentially more accurate solution than traditional methods.
  • Fine-tuning is key for specialized tasks: While zero-shot models show promise, fine-tuning LLMs on domain-specific data (like patient records) significantly boosts their performance for tasks like record linkage.
  • Blocking efficiency is a major win: The fine-tuned blocking model achieved a remarkable 92% reduction in candidate pairs, making the overall process much more efficient.
  • Reasoning models are computationally expensive: While models like DeepSeek-R1 show reasoning capabilities, their high computational cost makes them impractical for large-scale record linkage tasks.
  • Hybrid approaches might be best for blocking: For structured patient records, a combination of traditional rule-based blocking and probabilistic methods might still be more robust than pure LLM-based blocking, especially when dealing with minor typos.

Key Results

  • Blocking: The fine-tuned RoBERTa blocking model achieved a 92% reduction in candidate pairs while maintaining near-perfect recall. An optimal balance was found with K=10 and a cosine similarity threshold of 0.75.
  • Matching (Fine-tuned): Mistral-7B achieved the best performance with only 6 incorrect predictions (FP + FN). Llama-3.1-8B and Llama-3.2-3B also performed very well with 8 and 9 incorrect predictions, respectively.
  • Matching (Zero-Shot): Mistral-Small-24B performed best with 55 incorrect predictions, followed by Llama-3.3-70B with 79 incorrect predictions.
  • DeepSeek-R1-70B: Achieved an F1 score of 0.855 on a challenging subset but took 26 hours to process, compared to 30 minutes for Llama-3.3-70B.

Limitations and Future Work

  • Blocking limitations: The embedding-based blocking approach struggled with minor typos, suggesting that character-level tokenizers or hybrid rule-based/probabilistic methods might be more robust for certain use cases.
  • Computational cost of reasoning models: The significant computational demands of reasoning LLMs like DeepSeek-R1 limit their practicality for large-scale record linkage.
  • Future Work: The authors suggest exploring methodologies that incorporate character-level tokenizers for improved robustness against typos in blocking. They also highlight the need to balance predictive accuracy with computational efficiency when selecting models for real-world applications.

Summary updated

Abstract

Objective: Healthcare data fragmentation presents a major challenge for linking patient data, necessitating robust record linkage to integrate patient records from diverse sources. This study investigates the feasibility of leveraging language models for automated patient record linkage, focusing on two key tasks: blocking and matching. Materials and Methods: We utilized real-world healthcare data from the Missouri Cancer Registry and Research Center, linking patient records from two independent sources using probabilistic linkage as a baseline. A transformer-based model, RoBERTa, was fine-tuned for blocking using sentence embeddings. For matching, several language models were experimented under fine-tuned and zero-shot settings, assessing their performance against ground truth labels. Results: The fine-tuned blocking model achieved a 92% reduction in the number of candidate pairs while maintaining near-perfect recall. In the matching task, fine-tuned Mistral-7B achieved the best performance with only 6 incorrect predictions. Among zero-shot models, Mistral-Small-24B performed best, with a total of 55 incorrect predictions. Discussion: Fine-tuned language models achieved strong performance in patient record blocking and matching with minimal errors. However, they remain less accurate and efficient than a hybrid rule-based and probabilistic approach for blocking. Additionally, reasoning models like DeepSeek-R1 are impractical for large-scale record linkage due to high computational costs. Conclusion: This study highlights the potential of language models for automating patient record linkage, offering improved efficiency by eliminating the manual efforts required to perform patient record linkage. Overall, language models offer a scalable solution that can enhance data integration, reduce manual effort, and support disease surveillance and research.
BibTeX
@misc{beheshti2025leveraging,
  author = {Mohammad Beheshti and Lovedeep Gondara and Iris Zachary},
  title = {{Leveraging Language Models for Automated Patient Record Linkage}},
  year = {2025},
  eprint = {2504.15261},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.15261},
}
Figure