Transformers and the representation of biomedical background knowledge

Oskar Wysocki, Zili Zhou, Paul O'Regan, Deborah Ferreira, Magdalena Wysocka, Donal Landers, André Freitas

Introduction

In cancer precision medicine, transformer-based AI models are increasingly deployed to interpret the clinical significance of genomic alterations. While these models achieve high performance on benchmarks, they often operate as "black boxes." This research investigates whether specialized models like BioBERT and BioMegatron truly "understand" underlying biological principles or merely exploit statistical patterns—a phenomenon known as "shortcut learning." By employing probing and unsupervised clustering, the authors reveal how internal semantic structures are formed and subsequently altered during task-specific fine-tuning.

Methodology and Tasks

The study evaluates three architectures: BERT-base (general domain), BioBERT, and BioMegatron (biomedical domain). These models were tested on two primary tasks derived from the CIViC (Clinical Interpretation of Variants in Cancer) database:

  1. Task 1 (Binary Relations): Identifying if a pair (e.g., drug-gene, drug-variant) exists in a known database.
  2. Task 2 (N-ary Relations): Classifying the clinical significance (Sensitivity/Response vs. Resistance) of a quadruple consisting of [variant]-[gene]-[disease]-[drug].
Overview of Task 1 (binary relations) and Task 2 (n-ary relations) evaluated in the study.

To investigate the models' internal representations, the researchers utilized a multi-pronged approach:

  • Probing: Training linear classifiers on frozen embeddings to determine if they encode entity types (e.g., distinguishing a drug from a gene).
  • Unsupervised Clustering: Applying UMAP and HDBSCAN/HAC to visualize how models naturally group biological entities.
  • Bias Analysis: Comparing performance on balanced versus imbalanced datasets to detect reliance on entity frequency.
Experimental workflow for (a) classification tasks and (b) probing/clustering analysis to interpret model internal representations.

Key Insights: The Impact of Fine-Tuning

A central finding is that fine-tuning acts as a "double-edged sword." While it optimizes models for specific classification tasks, it simultaneously degrades their general understanding of entity types. Probing experiments showed that pre-trained models maintain distinct, "pure" clusters for different entity types. However, after fine-tuning, these clusters become overlapping and "corrupted" as the model reconfigures its internal space to prioritize task-specific labels.

Interestingly, pre-trained models exhibit emergent biological logic. Without explicit supervision, genes were found to cluster according to biological function, such as grouping members of the PI3K/AKT/mTOR signaling pathway. This suggests that large-scale pre-training on biomedical corpora does capture meaningful background knowledge.

Results and Performance

  • Classification Accuracy: Both BioBERT and BioMegatron performed well on Task 2 (AUC > 0.8), significantly outperforming a K-Nearest Neighbors (KNN) baseline.
  • Knowledge Degradation: Probing accuracy for identifying entity types dropped from ~94% in pre-trained BioBERT to ~73% after fine-tuning for Task 1.
  • The Frequency Trap: Models showed high sensitivity to data imbalance. If a drug appeared frequently in "True" associations during training, the model tended to label any pair containing that drug as "True," regardless of biological validity. This bias even led to the misclassification of well-established clinical facts (Level A evidence) when those entities were underrepresented in the training data.

Conclusion and Future Directions

The study highlights a critical safety concern: the tendency of transformers to prioritize statistical shortcuts over biological logic during fine-tuning. Future work should focus on developing fine-tuning techniques that preserve the model's underlying semantic structure (e.g., through partial freezing) and integrating structured knowledge sources like ontologies directly into the architecture to provide "safety rails" against statistical biases.

Summary updated

BibTeX
@article{wysocki2023transformers,
  author = {Oskar Wysocki and Zili Zhou and Paul O'Regan and Deborah Ferreira and Magdalena Wysocka and Donal Landers and André Freitas},
  title = {{Transformers and the representation of biomedical background knowledge}},
  journal = {Comput. Linguistics},
  year = {2023},
  doi = {10.1162/coli\_a\_00462},
}
Figure