TL;DR
LusoClin is a newly released dataset of over 54,000 synthetic clinical notes in European Portuguese, designed to enable medical Information Retrieval (IR) research without violating patient privacy. By fine-tuning a Llama 3.2 (3B) model on existing anonymized data, the researchers have created a high-utility, privacy-preserving resource for a traditionally low-resource language.
What Problem Does This Paper Address?
Advancing medical AI—specifically tasks like searching clinical notes or identifying patient cohorts—requires massive amounts of text. However, real clinical notes are protected by strict privacy regulations (like GDPR), making them nearly impossible for researchers to access or share.
This "data drought" is particularly severe for European Portuguese. While some resources exist for English or even Brazilian Portuguese, European Portuguese remains a low-resource language in the clinical domain. Researchers are often stuck in a catch-22: they need data to build models, but they can't get the data because of the very privacy concerns those models might eventually help address. LusoClin aims to bridge this gap by providing "fake" data that looks, acts, and computes like the real thing.
Methods
The authors developed a multi-stage generation pipeline to ensure both clinical realism and linguistic accuracy:
- Fine-tuning: They started with Meta’s Llama 3.2 (3B) Instruct model. To ground it in clinical language, they fine-tuned it using QLoRA (Quantized Low-Rank Adaptation) on two existing datasets: SemClinBr (Brazilian Portuguese) and CLINpt (European Portuguese).
- Prompt Engineering: The team iteratively refined prompts to force the model to act as a clinician. They included explicit constraints to avoid "LLM-isms" like repetitive phrasing or section headers, ensuring the output resembled unstructured "running text" typical of clinical encounters.
- Dialect Filtering: Since the training data included Brazilian Portuguese, the initial outputs were a mix of dialects. The authors used a BERT-based classifier (PtVId) to identify and retain only notes that met the criteria for European Portuguese (PT-PT), manually correcting common Brazilian terms to their European equivalents.
- Automatic Annotation: To make the dataset useful for downstream tasks, they used MediAlbertina, a specialized BERT model, to automatically tag clinical entities like diagnoses, symptoms, and medications.
Datasets
- SemClinBr: 1,000 Brazilian Portuguese clinical notes used for initial fine-tuning and few-shot examples.
- CLINpt: 281 European Portuguese neurology cases used to provide local linguistic grounding.
- LusoClin (The Result): A final collection of 54,518 synthetic notes categorized into four types: admission notes, ambulatory notes, discharge summaries, and nursing notes.
Key Insights and Ideas
- The Dialect Hurdle: One of the most interesting technical challenges was purging Brazilian Portuguese influence. Even after fine-tuning on European data, the model initially produced only ~32% PT-PT notes. It took targeted TF-IDF analysis and manual replacement of salient terms to boost this to 57%.
- Privacy by Design: The authors used Hausdorff distance and cosine similarity to prove that the synthetic notes aren't just "memorized" versions of the training data. The similarity scores were consistently low (~0.05), well below the 0.5 threshold typically used to signal privacy risks.
- The "Uncanny Valley" of Clinical Notes: In human evaluations, medical experts were surprisingly skeptical. Not only did they correctly identify most synthetic notes, but they also labeled 67% of real clinical notes as synthetic (see Table 2 below). This suggests that real clinical notes are often so messy or idiosyncratic that they "feel" fake to evaluators applying strict realism criteria.
Key Results
- Fidelity: Automatic metrics like Perplexity showed that LusoClin notes are actually easier to predict than real notes (68.32 vs 82.57), suggesting they are more standardized and medically coherent than the often-chaotic real-world equivalents.
- Diversity: t-SNE visualizations showed that while there is only partial overlap between LusoClin and real datasets, the synthetic data captures a comparable level of diversity, making it a robust proxy for multi-specialty clinical environments.
- Utility: The dataset successfully captures clinically relevant language patterns. While humans can spot "medical errors" or "non-sensical" content in about 17% of the justifications for calling a note synthetic, the overall structure and entity distribution (diagnoses, symptoms, etc.) align closely with real-world data.
Limitations and Future Work
The authors are transparent about the fact that LusoClin is a complementary resource, not a total replacement for real data.
- Medical Logic: Some notes still contain medical inconsistencies or "hallucinations" that a trained eye can catch.
- Dialect Purity: While the filtering was successful, the reliance on Brazilian source data remains a bottleneck for generating pure European Portuguese text.
- Future Directions: The team plans to explore more sophisticated "knowledge-infused" prompting to reduce medical errors and expand the dataset to cover more specific medical sub-specialties.
LusoClin is now publicly available, providing a vital sandbox for researchers to build the next generation of Portuguese medical AI.
Summary updated