Introduction
In human psychology, the Dunning-Kruger Effect (DKE) describes a cognitive bias where individuals with low competence in a domain overestimate their abilities. As Large Language Models (LLMs) become integral to software engineering, a critical question arises: do these models "know what they know"? This paper investigates whether LLMs exhibit a human-like DKE in coding tasks, particularly when operating in "low-resource" regimes like rare programming languages.
Methodology: Measuring Meta-Cognition
The researchers evaluated six prominent LLMs—including GPT-4o, GPT-o1, DeepSeek-Distill, Mistral, and the Phi family—across 37 programming languages. To quantify the DKE, they compared Actual Performance (AP), measured via multiple-choice questions (MCQA) from CodeNet, against Perceived Performance (PP).
Perceived Performance was captured using two distinct methods:
- Absolute Confidence: The model provides a raw confidence score (0-1) for its answer.
- Relative Confidence: A more robust pairwise comparison where the model identifies which of two questions it is more confident in. These preferences are aggregated into a scalar score using Elo and TrueSkill algorithms.
The Competence Gap
The study found a statistically significant correlation (Spearman’s $\rho = 0.775$) between low performance and high overestimation. Smaller models like Mistral and Phi-3 exhibit a massive disparity, often perceiving their performance to be near 80-90% even when their actual accuracy falls below 25%.
As shown in the figure above, state-of-the-art models like GPT-4o show much tighter alignment between their confidence and actual results. This suggests that as models become more "intelligent," they gain a better meta-cognitive sense of their own limitations.
The Rarity Trap
The DKE is most pronounced in rare programming languages such as COBOL, Prolog, or Ceylon. In these low-resource domains, training data is sparse, leading models to be "unskilled and unaware." Conversely, in high-resource languages like Python or JavaScript, models are better calibrated and occasionally even underconfident.
The language-wise analysis reveals that while actual performance generally increases with language popularity, perceived performance remains high across the board for weaker models, creating a significant "overconfidence gap" in the tail of the distribution.
Key Insights
- AI Mirrors Human Bias: LLMs don't just make mistakes; they follow the specific human pattern of being most overconfident when they are least competent.
- Relative vs. Absolute: Asking a model to compare its confidence between tasks yields a much more accurate picture of its internal state than asking for a raw percentage.
- Specialization Paradox: Interestingly, models specialized (fine-tuned) on a single domain showed a stronger DKE than generalist models, suggesting that narrow training might inflate a model's "ego" without providing the broad context needed for accurate self-assessment.
Conclusion and Future Work
The presence of the Dunning-Kruger Effect in LLMs raises critical questions about trust in AI-assisted software engineering. While the study is currently limited to programming and MCQA formats, it highlights the need for training strategies that reward accurate self-assessment. Future research should explore if this effect persists in other domains like medicine or law and investigate whether it is an inherent property of transformer architectures or a byproduct of training data distributions.
Summary updated