When evaluating models that extract structured knowledge from text or graphs, the standard practice is to measure their outputs against a human-annotated "gold standard" or reference database. However, this setup relies on a dangerous assumption: the Closed-World Assumption (CWA), which posits that any fact not explicitly recorded in the reference database is false.
Because real-world knowledge bases are inherently incomplete, models are frequently penalized for discovering valid new facts that were simply unrecorded. This fundamental flaw means that what automated benchmarks report as a "false positive" or an extraction error is often a genuine discovery.
The Core Dilemma of Incomplete Ground Truth
The tension between static evaluation sets and dynamic, real-world data creates a systematic measurement gap. As Ni Lao et al. observe in Reading The Web with Learned Syntactic-Semantic Inference Rules, evaluating knowledge extraction systems against a fixed reference creates a pessimistic bias because it penalizes the system for discovering valid, novel information missing from the original ground truth.
This problem stems from a failure to differentiate between factual falsehood and missing data. Volker Tresp et al. emphasize this in Link Prediction in Multi-relational Graphs using Additive Models, noting that treating missing triples as negative evidence fails to distinguish between facts that are false and facts that are simply missing from the knowledge base. As Grave argues in A convex relaxation for weakly supervised relation extraction, this incompleteness causes valid extractions to be incorrectly flagged as errors, meaning that the actual precision of modern models is likely higher than current benchmarks suggest.
The Penalty on Advanced and Agentic Models
The closed-world evaluation bias becomes particularly severe as extraction models grow more capable. When models leverage extensive external knowledge or advanced reasoning capabilities, their ability to find comprehensive information outpaces the static benchmarks used to test them.
For instance, Ananya Ananya et al. demonstrate in Towards Harnessing Large Language Models as Autonomous Agents for Semantic Triple Extraction from Unstructured Text that traditional evaluation metrics fail to account for the high-recall capabilities of language model agents. Because these agents extract more comprehensive data than the original gold standard, they face a performance penalty for correct but uncatalogued predictions, creating a stark discrepancy between quantitative benchmark scores and qualitative accuracy.
Similarly, Mehwish Alam et al. argue in Towards Semantically Enriched Embeddings for Knowledge Graph Completion that the closed-world assumption creates an explicit bias against models with extensive external knowledge, unfairly marking them as incorrect for identifying valid but unrecorded relationships.
Pervasiveness Across Tasks and Modalities
This evaluation gap is not isolated to standard relation extraction; it manifests across coreference resolution, entity typing, biomedical text mining, and even multi-modal systems:
- Document-Level Relation Extraction (DocRE): Popular benchmarks like DocRED suffer from significant under-annotation. Lu Xu et al. in Revisiting DocRED - Addressing the False Negative Problem in Relation Extraction and Qingyu Tan et al. in Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation show that models are routinely penalized for identifying correct relations overlooked by annotators. Correcting these omissions reveals that existing architectures possess much higher reasoning capabilities than previously measured.
- Biomedical Extraction: In specialized domains, human curators frequently miss valid interpretations. Sampo Pyysalo et al. in Overview of the ID, EPI and REL tasks of BioNLP Shared Task 2011 and Chrysoula Zerva et al. in Event Extraction in pieces: Tackling the partial event identification problem on unseen corpora note that manual review reveals a notable portion of system errors are actually correct extractions absent from the initial reference data.
- Entity Typing and Coreference: In Corpus-level Fine-grained Entity Typing, Yadollah Yaghoobzadeh et al. explain that when a model correctly identifies an entity type absent from the reference data, it is penalized as a false positive. Xuanyue Yang et al. identify a similar misalignment in Linguistically-Inspired Neural Coreference Resolution, where valid entity mentions are omitted by human annotators.
- Vision and Mapping: The bias extends beyond text. Boxu Chen et al. demonstrate in Seeing It or Not? Interpretable Vision-aware Latent Steering to Mitigate Object Hallucinations that standard metrics misidentify correct, visually grounded descriptions as hallucinations if the objects mentioned are not included in the human-annotated ground truth. In geographic tasks, Rui Zhang et al. in Map Generation from Large Scale Incomplete and Inaccurate Data Labels find that models are penalized for identifying real-world objects omitted from reference labels.
- Molecular Interactions: Validation databases for molecular interactions frequently lack a comprehensive catalog of all true biological relationships. Zhang notes in Ensemble Methods for MiRNA Target Prediction from Expression Data that this incompleteness leads to skewed performance metrics and an underestimation of a model's predictive capabilities when true positives are not yet experimentally verified.
Quantifying the Discrepancy
Just how much does this bias distort our understanding of model performance? Manual human validation and error analysis reveal that the gap is massive.
In the context of knowledge graph completion, the authors of SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models conducted human evaluations and discovered that nearly half of the predicted errors in high-performing models actually represented genuine discoveries rather than mistakes. This indicates that standard metrics severely underestimate model accuracy because the ground-truth graphs are missing so many valid facts.
Similarly, Sebastian Blohm et al. show in Harvesting Relations from the Web - Quantifiying the Impact of Filtering Functions that evaluating web-scale extraction against static gold standards underestimates precision by a significant factor compared to manual human validation, because the web contains a vast number of correct facts not yet represented in reference datasets.
Evolving Beyond the Closed-World Assumption
To build a more accurate picture of model capabilities, researchers are developing evaluation protocols and training paradigms that explicitly account for ground truth incompleteness:
- Ranking-Based Metrics: Instead of treating evaluation as a binary classification task under a closed-world assumption, ranking-based metrics evaluate the relative position of true matches among hard negatives. As Yuan He et al. demonstrate in Machine Learning-Friendly Biomedical Datasets for Equivalence and Subsumption Ontology Matching, this approach provides a more accurate assessment when reference sets are noisy or incomplete, preventing the unfair penalization of high-recall models.
- Open-World Evaluation Spaces: To avoid evaluation artifacts where models succeed merely by filtering statistically rare entities, Weihua Hu et al. in OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs advocate for ranking predictions against the entire entity space, better reflecting the true difficulty of open-world knowledge graph completion.
- Adversarial False Negative Mining: During training, the same incompleteness can corrupt distantly supervised models. To combat this, Botao Yu et al. introduce Knowing False Negatives: An Adversarial Training Method for Distantly Supervised Relation Extraction, which uses adversarial networks to "mine" hidden positives from the negative data pool, ensuring that models are not penalized during training or evaluation for identifying true relations absent from the reference knowledge base.
Key Foundational Work
Harvesting Relations from the Web - Quantifiying the Impact of Filtering Functions (2007) — contributed 3 concept elements to this direction.