Probing the Knowledge Boundary: An Interactive Agentic Framework for Deep Knowledge Extraction

Yuheng Yang, Siqi Zhu, Tao Feng, Ge Liu, Jiaxuan You

Large Language Models (LLMs) are frequently treated as compressed knowledge bases, yet quantifying exactly what they "know" remains a significant challenge. Traditional evaluations rely on static benchmarks like MMLU, which suffer from data contamination and passive sampling. Furthermore, LLMs exhibit "output inertia"—a tendency to default to high-probability, generic responses when prompted for broad information.

To overcome these limitations, researchers have developed an interactive, agentic framework that treats LLMs as environments to be actively mined, pushing them to their "extractable knowledge frontier."

Methodology: The Extraction Framework

The proposed framework consists of two primary modules designed to systematically map out a model's knowledge until it reaches "saturation" (the point where it stops producing new information):

1. Agentic Exploration Policies

Moving beyond simple prompting, the system deploys agents to extract discrete, verifiable statements called "Knowledge Atoms." The most effective strategy identified is the Recursive Taxonomy Explorer. This approach forces the model to decompose a broad topic into a hierarchical tree of sub-fields before exhaustively extracting facts at the "leaf" nodes.

2. The Knowledge Processor

To ensure the extracted data is valid and unique, a three-stage pipeline is utilized:

  • Vector Filtering: Employs embeddings to rapidly merge near-identical statements.
  • LLM Adjudication: Utilizes a high-reasoning model (DeepSeek-V3.1) to resolve subtle semantic overlaps that simple vector math might miss.
  • Domain Relevance Auditing: Filters statements based on Bloom’s Taxonomy, discarding overly generic claims (e.g., "Deep learning is useful").

Experimental Setup

The framework was tested across three technical domains derived from the ICML official topic taxonomy: Deep Learning (Engineering-focused), Machine Learning Systems (Systems-focused), and Probabilistic Methods (Theory-focused). Evaluated models included the Llama-3.1 family (8B, 70B, 405B), Qwen-2.5 (Instruct and Coder variants), and DeepSeek-R1-Distill-7B.

Key Findings

  • Structure Drives Depth: Naive sequential probing (e.g., repeatedly asking "What else?") plateaus quickly. The Recursive Taxonomy strategy extracted 6x more knowledge than baseline methods, demonstrating that LLMs require structural guidance to access long-tail parametric memory.
  • The Knowledge Scaling Law: Parameter count strongly correlates with knowledge volume. For instance, Llama-3.1 405B achieved a 39.0% recall of the total "knowledge union," compared to 27.3% for the 8B model. However, diminishing returns appear at extreme scales; the jump from 8B to 70B yielded a +9.6% recall increase, while 70B to 405B added only +2.1%.
  • The Pass@1 vs. Pass@k Trade-off: Specialized models (like Qwen-Coder) exhibit high initial accuracy (Pass@1) in their target domains but suffer rapid degradation during sustained extraction—dropping in accuracy at 3.7x the rate of general models. They also suffer from "catastrophic forgetting" in non-target domains. General-purpose models maintain stability over a much longer extraction horizon.
  • Pretraining Data Shapes Profiles: Even at similar scales (~7B parameters), model families display distinct knowledge profiles. Llama models excelled in practical engineering domains, whereas Qwen models dominated in theoretical and mathematical areas, directly reflecting their pretraining data diets.

Limitations and Future Directions

The authors note that these measurements are pipeline-dependent, capturing what can be extracted via this specific method rather than the absolute totality of a model's knowledge.

Additionally, the current reliance on an LLM Judge for deduplication, while showing high human agreement (89.6%), could be improved. Future iterations aim to incorporate retrieval-based fact-checking against external sources (like Wikipedia or textbooks) to mitigate hallucination risks. Expanding the framework's testing beyond objective technical domains into the humanities or subjective fields remains an open challenge.

Summary updated

Abstract

Large Language Models (LLMs) can be seen as compressed knowledge bases, but it remains unclear what knowledge they truly contain and how far their knowledge boundary extends. Existing benchmarks are mostly static and provide limited support for systematic knowledge probing. In this paper, we propose an interactive agentic framework to systematically extract and quantify the knowledge of LLMs. Our method includes four adaptive exploration policies to probe knowledge at different granularity. To ensure the quality of extracted knowledge, we introduce a three-stage knowledge processing pipeline that combines vector-based filtering to remove strict duplicates, LLM-based adjudication to resolve ambiguous semantic overlap, and domain relevance auditing to retain valid knowledge units. Through extensive experiments, we find that Recursive Taxonomy is the most effective exploration strategy. We also observe a clear knowledge scaling law, where larger models consistently recover more knowledge. In addition, we identify a Pass@1 versus Pass@k trade-off: domain-specialized models achieve higher initial accuracy but experience rapid degradation, while general-purpose models maintain stable performance over extended extraction. Finally, our results show that differences in training data composition lead to distinct and measurable knowledge profiles across model families, reflecting how pretraining shapes each model's parametric knowledge.
BibTeX
@inproceedings{yang2026probing,
  author = {Yuheng Yang and Siqi Zhu and Tao Feng and Ge Liu and Jiaxuan You},
  title = {{Probing the Knowledge Boundary: An Interactive Agentic Framework for Deep Knowledge Extraction}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=gY9mL1HW2M},
}
Figure