Evaluating Grammatical Well-Formedness in Large Language Models: A Comparative Study with Human Judgments

Zhuang Qiu, Xufeng Duan, Zhenguang CaiACL 2024

TL;DR

This paper investigates whether Large Language Models (LLMs) like ChatGPT and Vicuna can accurately judge the grammatical well-formedness of English sentences, a key aspect of linguistic knowledge. By comparing LLM judgments to human judgments across three experiments with varying response formats, the study finds significant alignment, suggesting LLMs possess a substantial amount of shared grammatical knowledge with humans, though notable differences in conservativeness and sensitivity to language origin persist.

What Problem Does This Paper Address?

The rapid advancement of Large Language Models (LLMs) has led to impressive performance across various linguistic tasks. This raises a fundamental question: to what extent do these models truly mirror human linguistic cognition and usage? Specifically, this paper focuses on "grammatical well-formedness," a crucial component of linguistic knowledge that allows us to distinguish between grammatically correct and incorrect sentences. While some argue LLMs process language fundamentally differently from humans, others suggest they might reflect genuine human linguistic processes. Existing research often relies on binary grammaticality judgments or focuses on specific syntactic phenomena. This study aims to provide a more comprehensive understanding by comparing LLM judgments with human judgments on a larger scale and across different response formats, exploring whether LLMs can accurately represent and evaluate grammatical correctness.

Methods

The researchers conducted three experiments to assess LLMs' grammatical judgment capabilities.

  • Experiment 1: This experiment used a binary judgment task, similar to previous work. ChatGPT and Vicuna were asked to classify English sentences as either "natural" or "unnatural."
  • Experiment 2: This experiment modified the response type to a four-point Likert scale (extremely unnatural to extremely natural), aiming for more sensitivity to graded grammaticality. The same stimuli as Experiment 1 were used.
  • Experiment 3: This experiment employed a sliding scale judgment task, ranging from 1 (extremely unnatural) to 100 (extremely natural), a method previously used by Lau et al. (2014, 2017).

In all experiments, LLM judgments were compared to human judgments. Statistical analyses included correlational analyses to measure the similarity between human and LLM ratings, and Bayesian mixed-effects logistic regression models to examine the influence of participant type (human, ChatGPT, Vicuna) and the origin language of the stimuli on judgments.

Datasets

The study utilized datasets originally collected by Lau et al. (2014, 2017). These datasets consist of English sentences of varying grammaticality, created through an automated procedure involving translation between English and four other languages (Norwegian, Spanish, Chinese, and Japanese) and then back to English. This process aimed to generate a ranked distribution of relative grammatical well-formedness.

  • Experiment 1 & 2: Used 2469 distinct English sentences.
  • Experiment 3: Used 248 unique sentences, a subset of the previous datasets.

No new benchmarks were introduced; instead, existing human judgment data from Lau et al. was used as a proxy for human grammaticality knowledge.

Key Insights and Ideas

  • LLMs and humans share significant grammatical knowledge: The strong correlations observed across all three experiments suggest that LLMs and humans generally agree on what constitutes a natural or unnatural sentence. This implies LLMs have learned a substantial amount about sentence structure and acceptability from their training data.
  • LLMs can be more "conservative" than humans: In binary and graded tasks, LLMs, particularly ChatGPT, tended to rate sentences as less natural than humans did, especially when those sentences were derived from languages other than English. This suggests LLMs might be less tolerant of deviations from what they've learned as "standard" English.
  • LLMs exhibit heterogeneity: While both ChatGPT and Vicuna showed alignment with human judgments, they differed in their behavior. ChatGPT's ratings varied more across language conditions, mimicking human patterns, whereas Vicuna's ratings were more stable, suggesting different internal processing mechanisms.
  • Language origin matters: The source language of the translated sentences significantly impacted judgments for both humans and LLMs, with sentences derived from languages typologically further from English often being rated as less natural. However, Vicuna showed an interesting tendency to rate these less natural sentences higher than humans did.
  • Response format influences results: The shift from binary to graded and sliding scale judgments revealed more nuanced patterns and highlighted differences in LLM sensitivity to grammaticality.

Key Results

  • Experiment 1 (Binary Judgment):
    • Strong correlation between human and ChatGPT judgments (r = 0.83).
  • Significant correlation between human and Vicuna judgments (r = 0.66).
  • ChatGPT was more likely to judge original English sentences as natural than humans.
  • Vicuna was less likely to judge original English sentences as natural than humans.
  • Human judgments decreased for sentences derived from languages other than English.
  • Experiment 2 (Four-Point Likert Scale):
    • Strong correlation between human and ChatGPT judgments (r = 0.84).
  • Significant correlation between human and Vicuna judgments (r = 0.49).
  • Human participants rated original English stimuli higher than ChatGPT and Vicuna.
  • ChatGPT's ratings mimicked human patterns across language conditions (English > Norwegian/Spanish > Chinese/Japanese).
  • Vicuna showed minimal variation in ratings across language conditions.
  • Experiment 3 (Sliding Scale):
    • Strong correlation between human and ChatGPT ratings (r = 0.81).
  • Significant correlation between human and Vicuna ratings (r = 0.72).
  • Human participants rated original English stimuli higher than ChatGPT and Vicuna.
  • LLMs generally rated translated sentences, especially from Chinese and Japanese, lower than humans.

Limitations and Future Work

  • Prompting vs. Probability: The study acknowledges that using prompting methods to elicit judgments might not be as accurate as using sentence probability measures. Future work could replicate the study using probability measures to gain a more comprehensive understanding of LLMs' linguistic capabilities.
  • Stimuli Size in Experiment 3: The smaller number of stimuli in Experiment 3 led to larger error terms in the mixed-effects model estimates, making variations less statistically robust compared to previous experiments.
  • "Conservativeness" of LLMs: The paper suggests LLMs are more "conservative" in their naturalness judgments, particularly for sentences derived from non-English languages. Further research could explore the underlying reasons for this conservativeness and how it relates to their "distributional knowledge" of sentence grammaticality.
  • Data-Driven vs. Cognitive Processes: The authors highlight that LLMs rely on statistical patterns from training data, while humans integrate diverse cognitive and contextual cues. Future work could focus on incorporating more diverse real-world language data and sophisticated cognitive models into LLM training to improve their linguistic adaptability and judgment accuracy.

Summary updated

BibTeX
@inproceedings{qiu2024evaluating,
  author = {Zhuang Qiu and Xufeng Duan and Zhenguang Cai},
  title = {{Evaluating Grammatical Well-Formedness in Large Language Models: A Comparative Study with Human Judgments}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure