Introspective Growth: Automatically Advancing LLM Expertise in Technology Judgment

Siyang Wu, Honglin Bao, Nadav Kunievsky, James A. Evans

Abstract

Large Language Models (LLMs) frequently struggle with complex technical tasks, leading to the assumption that they lack the necessary domain-specific knowledge. This research challenges that assumption by investigating the Missing vs. Unused Knowledge Hypothesis. Focusing on the intricate domain of patent law, we introduce a "self-talk" framework where models generate and answer their own clarifying questions. Our findings reveal that much of the failure in technical judgment is a "deployment bottleneck"—knowledge is present but inactive. We demonstrate that internal activation can significantly improve performance, often matching the gains provided by external scientific retrieval.

Introduction

When an LLM fails to solve a technical problem, the standard remedy is to provide more data through fine-tuning or Retrieval-Augmented Generation (RAG). However, this approach ignores the possibility that the model already contains the required information but cannot access it during standard inference. This paper explores whether LLMs suffer from a gap in knowledge or a gap in application.

The authors use patent understanding as a testbed. Patents are notoriously difficult due to their "strategically obfuscated" language, designed to be broad yet legally distinct. Distinguishing between two highly similar patents requires deep conceptual understanding rather than simple fact recall.

Methodology: Activating Latent Knowledge

To distinguish between "lay-in" knowledge (stored but unused) and "working" knowledge (applied), the researchers tested models under three conditions:

  1. Baseline: Direct prompting to judge if two patents are identical or distinct.
  2. Self-QA (Internal Activation): The model generates its own clarifying questions and answers them using internal parameters before making a final judgment.
  3. Scientific RAG (External Supply): The model's questions are answered using external scientific papers from arXiv.

The study utilized a massive dataset of over 1.3 million ICT patents and 177,000 Biomedical patents, all granted post-2015 to ensure they were not part of the models' initial training sets.

Results: Unlocking the Deployment Bottleneck

The baseline results were poor; for instance, Llama 3.1 8B incorrectly classified approximately 70% of distinct patent pairs as identical. However, the introduction of "self-talk" (Self-QA) led to significant accuracy improvements. This suggests that the information needed to differentiate the patents was already present in the model's latent space but required structured "nudging" to be retrieved.

PCA visualization comparing the semantic space of internal 'self-talk' vs. external scientific knowledge.

As shown in the PCA analysis above, the semantic space covered by the model's internal self-talk (orange) largely overlaps with the space covered by external scientific knowledge (blue). This visual evidence supports the claim that internal "lay-in" knowledge is nearly as comprehensive as external databases for these technical tasks.

Discussion: Model Scale and the Overconfidence Trap

The research highlighted several nuances in how different model architectures handle technical reasoning.

Cross-Scale Complementarity

Surprisingly, smaller models (1B to 3B parameters) were often more effective at generating foundational questions. While larger models like GPT-4o mini are more capable reasoners, they sometimes generate overly complex or brittle questions that derail the logic chain.

Response Bias and Confidence

Models exhibited significant "response bias" depending on how questions were framed. For example, Mistral 7B showed a strong tendency to agree with the prompt's premise regardless of the underlying facts. Furthermore, larger models often displayed higher confidence in their incorrect answers than in their correct ones.

Model performance and confidence levels when judging patent differences under different prompt framings.

The figure above illustrates this discrepancy. When asked if patents are "different" (a), models like Llama 3.1 8B show high accuracy but varying confidence. However, when the question is flipped to ask if they are the "same" (b), the "Vote=Yes" (wrong) rate spikes, revealing a reliance on superficial heuristics rather than robust reasoning.

Conclusion

The "deployment bottleneck" is a critical hurdle for LLM expertise. By using self-talk to activate latent knowledge, we can advance model performance without necessarily increasing the size of the training set or the complexity of external retrieval systems. Future work will focus on agentic retrieval strategies and hybrid architectures where small models scaffold the reasoning for larger, more powerful counterparts.

Summary updated

Abstract

While large language models (LLMs) excel at factual recall, the real challenge lies in knowledge application. A gap persists between their ability to answer complex questions and their effectiveness in performing tasks that require that knowledge. We investigate this gap using a patent classification problem that requires deep conceptual understanding to distinguish semantically similar but objectively different patents written in dense, strategic technical language. We find that LLMs often struggle with this distinction. To diagnose the source of these failures, we introduce a framework that decomposes model errors into two categories: missing knowledge and unused knowledge. Our method prompts models to generate clarifying questions and compares three settings -- raw performance, self-answered questions that activate internal knowledge, and externally provided answers that supply missing knowledge (if any). We show that most errors stem from failures to deploy existing knowledge rather than from true knowledge gaps. We also examine how models differ in constructing task-specific question-answer databases. Smaller models tend to generate simpler questions that they, and other models, can retrieve and use effectively, whereas larger models produce more complex questions that are less effective, suggesting complementary strengths across model scales. Together, our findings highlight that shifting evaluation from static fact recall to dynamic knowledge application offers a more informative view of model capabilities.
BibTeX
@misc{wu2025introspective,
  author = {Siyang Wu and Honglin Bao and Nadav Kunievsky and James A. Evans},
  title = {{Introspective Growth: Automatically Advancing LLM Expertise in Technology Judgment}},
  year = {2025},
  eprint = {2505.12452},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.12452},
}
Figure