Abstract
Recent work in machine learning increasingly attributes human-like capabilities such as reasoning or theory of mind to large language models (LLMs) on the basis of benchmark performance. This paper examines this practice through the lens of construct validity, understood as the problem of linking theoretical capabilities to their empirical measurements. It contrasts three influential frameworks: the nomological account developed by Cronbach and Meehl, the inferential account proposed by Messick and refined by Kane, and Borsboom's causal account. I argue that the nomological account provides the most suitable foundation for current LLM capability research. It avoids the strong ontological commitments of the causal account while offering a more substantive framework for articulating construct meaning than the inferential account. I explore the conceptual implications of adopting the nomological account for LLM research through a concrete case: the assessment of reasoning capabilities in LLMs.
Introduction: The Validity Gap in AI Evaluation
As Large Language Models (LLMs) demonstrate increasingly sophisticated outputs, the AI research community has begun to describe model performance using psychological terminology such as "reasoning," "theory of mind," or "understanding." However, these claims often rest on a precarious foundation. When a model passes a specific benchmark, researchers frequently conclude that the model possesses the underlying capability the benchmark was designed to test. This leap from performance to capability is a question of construct validity: the degree to which a test measures the abstract, unobservable quality (the "construct") it claims to measure.
In the absence of a rigorous framework for construct validity, AI evaluation faces two primary risks. First is construct underrepresentation, where a benchmark captures only a narrow slice of a complex capability, leading to overgeneralized claims. Second is the risk of anthropomorphism and data contamination, where models may achieve high scores through pattern matching or memorization of training data rather than the exercise of the target capability.
Theoretical Frameworks for Construct Validity
To address these challenges, we can look to psychometrics, which offers three primary accounts of how to link abstract constructs to empirical measurements:
- The Causal Account: This view, championed by Borsboom, posits that the construct must exist as a real entity that causes the observed behavior. In the context of LLMs, this would require proving that a specific internal mechanism (like a "reasoning circuit") exists and is responsible for the output. Given our current limited understanding of LLM internals, this account is often too demanding for immediate use.
- The Inferential Account: Proposed by Messick and Kane, this framework focuses on the validity of the uses and consequences of test scores. While useful for policy and application, it is often too broad and pragmatic to provide a substantive definition of what a capability actually is.
- The Nomological Account: Developed by Cronbach and Meehl (1955), this account defines a construct by its position within a "nomological network"—a web of laws and statistical relationships. A construct is defined not by a single score, but by how it relates to other tasks, traits, and theoretical predictions.
The Nomological Network as a Foundation for LLM Research
The nomological account is particularly well-suited for LLM research because it allows us to define capabilities through empirical observation without requiring immediate "mechanistic" proof. In this view, a capability like "reasoning" is a "knot" in a web of correlations.
To establish a capability within this framework, researchers must perform both convergent and discriminant analysis. Convergent analysis requires showing that a model’s performance on a reasoning task correlates with its performance on other, theoretically related tasks (e.g., logical deduction and mathematical problem-solving). Discriminant analysis requires showing that the reasoning score is not merely a proxy for an unrelated trait, such as word frequency or simple sequence memorization. If a "reasoning" score is perfectly correlated with the model's training data exposure, the "knot" of reasoning disappears, replaced by the construct of "memory."
A Proposed Workflow for Benchmark Design
Moving toward a nomological approach requires a shift in how benchmarks are developed. The paper proposes a four-step workflow for more rigorous evaluation:
- Define the Nomological Network: Map the target capability (e.g., reasoning) against existing cognitive theories, such as the Cattell–Horn–Carroll (CHC) theory, which provides a structured model of human cognitive architecture.
- Design Task Benchmarks: Create a suite of tests that target different components of the network.
- Verify Statistical Relationships: Analyze whether the model’s performance across these tests aligns with the predicted theoretical correlations.
- Iterative Refinement: If the empirical data contradicts the theory, researchers must determine whether the benchmark is flawed or if the theoretical understanding of "LLM reasoning" needs to be revised.
Challenges: The Substrate and Duhem-Quine Problems
Adopting psychometric frameworks for AI is not without difficulty. The substrate problem acknowledges that LLM "cognition" may be fundamentally different from human cognition. While human theories like CHC provide a principled starting point, LLM-specific nomological networks may eventually diverge as we discover unique model behaviors.
Furthermore, the Duhem-Quine problem suggests that when a model fails a test, it is difficult to isolate whether the failure lies in the model's lack of capability or in a flaw in the test itself. Resolving these ambiguities will require an iterative process of theoretical refinement and, eventually, insights from mechanistic interpretability to ground these statistical webs in physical model circuits.
Conclusion
If the AI community continues to use human-centric labels to describe machine capabilities, it must adopt the rigorous standards that make those labels scientifically meaningful. By adopting nomological networks, researchers can move beyond isolated performance metrics toward a more robust, theoretically grounded understanding of what LLMs can truly do.
Summary updated