Beyond Surface Structure: A Causal Assessment of LLMs' Comprehension Ability

Yujin Han, Lei Xu, Sirui Chen, Difan Zou, Chaochao Lu

Abstract

Large language models (LLMs) have shown remarkable capability in natural language tasks, yet debate persists on whether they truly comprehend deep structure (i.e., core semantics) or merely rely on surface structure (e.g., presentation format). Prior studies observe that LLMs' performance declines when intervening on surface structure, arguing their success relies on surface structure recognition. However, surface structure sensitivity does not prevent deep structure comprehension. Rigorously evaluating LLMs' capability requires analyzing both, yet deep structure is often overlooked. To this end, we assess LLMs' comprehension ability using causal mediation analysis, aiming to fully discover the capability of using both deep and surface structures. Specifically, we formulate the comprehension of deep structure as direct causal effect (DCE) and that of surface structure as indirect causal effect (ICE), respectively. To address the non-estimability of original DCE and ICE -- stemming from the infeasibility of isolating mutual influences of deep and surface structures, we develop the corresponding quantifiable surrogates, including approximated DCE (ADCE) and approximated ICE (AICE). We further apply the ADCE to evaluate a series of mainstream LLMs, showing that most of them exhibit deep structure comprehension ability, which grows along with the prediction accuracy. Comparing ADCE and AICE demonstrates closed-source LLMs rely more on deep structure, while open-source LLMs are more surface-sensitive, which decreases with model scale. Theoretically, ADCE is a bidirectional evaluation, which measures both the sufficiency and necessity of deep structure changes in causing output variations, thus offering a more comprehensive assessment than accuracy, a common evaluation in LLMs. Our work provides new insights into LLMs' deep structure comprehension and offers novel methods for LLMs evaluation.

Introduction

The question of whether Large Language Models (LLMs) possess true semantic understanding or are merely "stochastic parrots" remains central to AI research. Critics often cite LLMs' sensitivity to surface-level changes—such as rephrasing a prompt—as evidence of a lack of deep comprehension. However, this sensitivity does not inherently preclude the existence of deep structural understanding. This study addresses this gap by employing causal mediation analysis to disentangle and quantify the influence of core semantics (deep structure) versus linguistic form (surface structure) on model outputs.

Methodology: Causal Mediation Analysis

We model the input $x$ as a combination of deep structure ($d$) and surface structure ($s$). To evaluate comprehension, we utilize Causal Mediation Analysis to measure the Direct Causal Effect (DCE) and the Indirect Causal Effect (ICE). Because $d$ and $s$ are often inseparable in natural language, we introduce Approximated DCE (ADCE) and Approximated ICE (AICE) as quantifiable surrogates.

The ADCE framework involves initial inference, generating intervention data through masking or rephrasing, and secondary inference to calculate causal effects.

These metrics are calculated through intervention strategies such as masking core semantic tokens or rephrasing questions using an auxiliary agent model. We evaluated 12 prominent LLMs across benchmarks covering mathematics (GSM8k), logic (Word Unscrambling), and commonsense reasoning.

Results and Discussion

Our analysis reveals several key findings regarding the nature of LLM comprehension:

Deep Comprehension and Scaling

Most evaluated LLMs demonstrate a positive ADCE, indicating that their predictions are primarily driven by deep structural understanding. We observed a strong linear correlation ($R^2 > 0.7$) between a model's accuracy and its ADCE, suggesting that as models scale and improve in performance, their reliance on deep structure increases.

Surface Sensitivity vs. Deep Structure

While models exhibit sensitivity to surface changes, their reliance on deep structure is far more critical for performance. As shown in the comparison for Llama-3 models, intervening only on surface structure leads to a minor decrease in accuracy. In contrast, intervening on both deep and surface structures causes a near-total collapse in performance, particularly in larger models like Llama-3-70b.

Accuracy comparison for Llama-3 models under surface-only vs. deep and surface structure interventions, highlighting the critical role of deep structure.

Closed-Source vs. Open-Source Models

Closed-source models like GPT-4o and Claude-3.5-Sonnet consistently show a higher reliance on deep structure compared to open-source models. While open-source models are more sensitive to surface-level variations, this sensitivity tends to decrease as model size increases. Additionally, Supervised Fine-Tuning (SFT) was found to "activate" a model's ability to utilize deep structure for specific tasks where it previously struggled.

Conclusion

This work introduces ADCE as a robust metric for evaluating true comprehension in LLMs. Unlike simple accuracy, which can be inflated by surface-level patterns or spurious correlations, ADCE measures the causal necessity and sufficiency of deep structure in generating correct outputs. Our findings provide strong evidence that modern LLMs are moving beyond surface-level mimicry toward genuine semantic comprehension.

Summary updated

Abstract

Large language models (LLMs) have shown remarkable capability in natural language tasks, yet debate persists on whether they truly comprehend deep structure (i.e., core semantics) or merely rely on surface structure (e.g., presentation format). Prior studies observe that LLMs' performance declines when intervening on surface structure, arguing their success relies on surface structure recognition. However, surface structure sensitivity does not prevent deep structure comprehension. Rigorously evaluating LLMs' capability requires analyzing both, yet deep structure is often overlooked. To this end, we assess LLMs' comprehension ability using causal mediation analysis, aiming to fully discover the capability of using both deep and surface structures. Specifically, we formulate the comprehension of deep structure as direct causal effect (DCE) and that of surface structure as indirect causal effect (ICE), respectively. To address the non-estimability of original DCE and ICE -- stemming from the infeasibility of isolating mutual influences of deep and surface structures, we develop the corresponding quantifiable surrogates, including approximated DCE (ADCE) and approximated ICE (AICE). We further apply the ADCE to evaluate a series of mainstream LLMs, showing that most of them exhibit deep structure comprehension ability, which grows along with the prediction accuracy. Comparing ADCE and AICE demonstrates closed-source LLMs rely more on deep structure, while open-source LLMs are more surface-sensitive, which decreases with model scale. Theoretically, ADCE is a bidirectional evaluation, which measures both the sufficiency and necessity of deep structure changes in causing output variations, thus offering a more comprehensive assessment than accuracy, a common evaluation in LLMs. Our work provides new insights into LLMs' deep structure comprehension and offers novel methods for LLMs evaluation.
BibTeX
@inproceedings{han2025beyond,
  author = {Yujin Han and Lei Xu and Sirui Chen and Difan Zou and Chaochao Lu},
  title = {{Beyond Surface Structure: A Causal Assessment of LLMs' Comprehension Ability}},
  booktitle = {International Conference on Learning Representations},
  year = {2025},
  url = {https://openreview.net/forum?id=gsShHPxkUW},
}
Figure