Abstract
Large Language Models (LLMs) are increasingly integrated into clinical workflows, yet their internal decision-making processes remain opaque. This study introduces SUFO, a diagnostic framework designed to analyze the internal feature dynamics of transformer models during fine-tuning on clinical notes. By comparing general-domain, mixed-domain, and domain-specific models, we demonstrate that while domain-specific pre-training provides relevant raw features, mixed-domain models (e.g., Clinical BioBERT) offer superior stability and performance, particularly in imbalanced clinical datasets. Our findings suggest that mixed-domain pre-training facilitates faster feature disambiguation and greater robustness against overfitting.
Introduction
The deployment of LLMs in high-stakes medical fields like oncology necessitates a move beyond simple accuracy metrics toward a deeper understanding of model interpretability. A critical gap exists in understanding how pre-training data distributions—ranging from general web text to specialized medical literature—influence the internal "feature space" during task-specific fine-tuning. Without this insight, developing clinical AI that is both reliable and safe remains a challenge. This paper addresses this gap by examining how different pre-training strategies affect the evolution of internal logic during the classification of prostate cancer pathology reports.
Methodology
The researchers developed the SUFO framework, which utilizes four analytical pillars:
- Supervised Probing: Assessing the utility of features prior to task-specific training.
- Unsupervised Similarity Analysis: Employing Representational Similarity Analysis (RSA) to track layer-wise conceptual shifts during fine-tuning.
- Feature Dynamics: Visualizing data clustering across training epochs and layers using Principal Component Analysis (PCA).
- Outlier Analysis: Investigating data points that deviate from established clusters with expert clinical input.
The study evaluated five models: General-domain (BERT, TNLR), Mixed-domain (BioBERT, Clinical BioBERT), and Domain-specific (PubMedBERT). These were tested on a private dataset of 2,907 UCSF prostate cancer pathology reports and validated using the public MedNLI dataset.
Results and Analysis
Performance and Robustness
As shown in the performance evaluation, mixed-domain models (BioBERT and Clinical BioBERT) achieved the highest average F1 scores, approximately 0.95-0.96. A key finding was the "PubMedBERT Paradox": while PubMedBERT possesses the most relevant raw features for medical tasks, it proved highly unstable on imbalanced data. For instance, on the Path-PG task, PubMedBERT’s performance dropped to an F1 of 0.770, whereas Clinical BioBERT maintained a robust 0.959.
Feature Space Dynamics and Similarity
Analysis of the models' internal feature spaces post-fine-tuning revealed extreme sparsity, with the first two principal components explaining roughly 95% of the data variance. This indicates that models aggressively discard non-essential information to focus on task-specific features.
Unsupervised similarity analysis (RSA) demonstrated that mixed-domain models disambiguate features faster than their general-domain counterparts. Clinical models began to clearly separate data classes around Epoch 6, whereas general models like BERT required until Epoch 9 to reach comparable clarity.
Discussion and Conclusion
The SUFO framework reveals that pre-training data distribution is a primary determinant of model robustness in clinical settings. Mixed-domain pre-training appears to provide a "sweet spot," combining the broad linguistic foundations of general text with the specialized knowledge of medical corpora to prevent the overfitting observed in purely domain-specific models. Furthermore, the identification of outliers through SUFO serves as a valuable tool for clinical quality control, highlighting inconsistencies or boundary cases in medical documentation.
Future research should explore whether these feature-space dynamics persist in larger generative models and investigate optimal data mixing strategies for complex tasks like medical reasoning and summarization.
Summary updated