Evaluating Vocabulary Usage in LLMs

Matthew Durward, Christopher ThomsonNAACL 2024

TL;DR

This paper investigates how Large Language Models (LLMs) like GPT and LLaMA use vocabulary differently from humans, particularly across news and creative writing. By analyzing text structure with Vocabulary-Management Profiles (VMPs) and identifying key words, they found distinct patterns that could inform how LLMs are used in education.

What Problem Does This Paper Address?

As LLMs become increasingly integrated into education, especially for language learning, a crucial question arises: do the texts generated by these models accurately reflect real-world language use, or do they exhibit a distinct, potentially "generic" style? This paper aims to understand if LLMs generate text with similar vocabulary patterns and structures as humans across different registers (news vs. creative writing). This is important because if LLM-generated materials differ significantly from authentic human text, their pedagogical value might be compromised, potentially leading to the propagation of biases or a reduction in linguistic diversity.

Methods

The study employs two primary methods:

  • Vocabulary-Management Profiles (VMPs): This technique visualizes the rate at which new vocabulary is introduced throughout a text. By using a "moving window" approach, VMPs can reveal structural differences in writing patterns and highlight local patterns in vocabulary use. The authors use a specific VMP 2.2 method, which calculates scores for new words and uses a formula for repeated words to track the introduction of new vocabulary over the text's progression.
  • Word Keyness Analysis: This method focuses on identifying words that are statistically significant within a specific corpus (e.g., human-written text) compared to another (e.g., LLM-generated text). The study uses two aspects of keyness:
  • Frequency: Measures how often a word appears.
  • Dispersion: Assesses how evenly or unevenly a word is distributed throughout a corpus. The researchers use Kullback-Leibler (KL) divergence to quantify both frequency and dispersion, providing a nuanced understanding of vocabulary differences.

Datasets

The study utilizes a subset of the DeepfakeTextDetect dataset. This dataset initially comprised eight different registers and text generated from 27 LLMs. The researchers created a specific subset for their analysis, including text from OpenAI's gpt-turbo-3.5 and Meta's LLaMA 65B models, along with their human-generated counterparts. Texts were selected to be between 400 and 500 tokens in length to ensure sufficient data for VMP analysis. The final corpus consisted of 120 texts for each unique register/source combination, totaling 720 texts.

Key Insights and Ideas

  • Human writing exhibits higher lexical diversity: Across both news and creative registers, human-generated texts generally show higher Type/Token ratios, indicating a broader range of vocabulary.
  • LLMs have distinct stylistic signatures: The study found that different LLMs (GPT and LLaMA 65B) and human writers produce texts with discernible vocabulary patterns, which can be captured by VMPs and keyness analysis.
  • Register differences are pronounced in human writing: Human texts show a stronger distinction between news and creative registers compared to LLM-generated texts, suggesting LLMs might struggle to capture these nuanced stylistic shifts as effectively.
  • LLMs tend towards more abstract language in news: In news contexts, LLMs appear to use longer, more abstract words, while humans tend to use more concrete, shorter words. This might be because LLMs avoid inventing specific facts, which are common in human news reporting.
  • Creative LLM output can be more optimistic: Human creative writing samples showed a propensity for darker, more emotional language (e.g., "bloody," "torture"), while GPT's creative output leaned towards more optimistic terms (e.g., "succeeded," "grateful").

Key Results

  • Statistical significance: Kruskal-Wallis tests revealed highly significant p-values (p < 0.00001) for distinctions among group sources and registers across various VMP characteristic features.
  • Effect sizes: The study identified significant effect sizes, particularly for the news register with a window size of 25. For the DN_Mean feature (lexical diversity), an effect size of 0.4394 highlighted a marked distinction between 65B and human-generated texts.
  • Self-similarity: DTW-based self-similarity scores indicated that human news texts were more self-similar than creative texts. Conversely, 65B and GPT sources showed less consistent self-similarity patterns across registers.
  • Word Keyness: In creative writing, human texts featured words related to profanity and darker themes, absent in GPT's output. LLMs like GPT used more optimistic language. In news, LLMs used more abstract words, while humans used more concrete terms.

Limitations and Future Work

  • Text selection: The authors acknowledge that register can be fluid, and variations within a register might not be fully captured by the data. They also note that achieving perfect token length balance across all samples is challenging, and truncating texts might omit unique attributes.
  • Default LLM configurations: The study used default LLM configurations. Future work could investigate the impact of adjusting parameters like temperature or top-p on text generation and vocabulary usage.
  • Future research directions: The authors envision combining VMPs with word keyness features to identify LLM-generated text samples that balance desired vocabulary with appropriate repetition, which could be beneficial for language learning. They also suggest that understanding LLM production limitations can guide future research to make LLMs more representative of human language.

Summary updated

BibTeX
@inproceedings{durward2024evaluating,
  author = {Matthew Durward and Christopher Thomson},
  title = {{Evaluating Vocabulary Usage in LLMs}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure