Abstract
The prevalent use of Large Language Models (LLMs) has necessitated studying their mental models, yielding noteworthy theoretical and practical implications. Current research has demonstrated that state-of-the-art LLMs, such as ChatGPT, exhibit certain theory of mind capabilities and possess relatively stable personality traits. However, cognitive process features, which differ significantly between Eastern (holistic) and Western (analytic) cultures, remain under-explored. In this research, we probed the cultural cognitive traits of ChatGPT using the Analysis-Holism Scale (AHS), Triadic Categorization Task (TCT), Dialectical Self Scale (DSS), and Self-construal Scale (SCS). Our findings indicate that while ChatGPT consistently tends towards Eastern holistic thinking in cognitive processes, it does not significantly lean towards the East or West in value judgments, often exhibiting a Western-leaning independent self-construal. We suggest these results stem from both the training paradigm and the training data in LLM development.
Introduction
As Large Language Models (LLMs) like ChatGPT become deeply integrated into societal functions, understanding their underlying "mental models" is critical. A key dimension of human psychology is the cultural divide in cognitive styles: Analytic thinking, common in Western cultures, focuses on discrete objects and formal logic; Holistic thinking, prevalent in Eastern cultures, emphasizes context, relationships, and the acceptance of contradiction (naive dialecticism). This study investigates where ChatGPT falls on this spectrum and whether its cognitive style is consistent across different psychological assessments.
Methods
We evaluated OpenAI’s GPT-3.5 and GPT-4 using four established psychological scales:
- Cognitive Process Measures: The Analysis-Holism Scale (AHS) and the Triadic Categorization Task (TCT), an implicit task where models group objects based on either category (analytic) or relationship (holistic).
- Value Judgment Measures: The Dialectical Self Scale (DSS) and the Self-Construal Scale (SCS), which measures independent vs. interdependent views of the self.
To ensure robustness, models were tested across multiple "temperatures" (randomness settings) and various prompt paraphrases. Results were compared against established human norms from psychological literature.
Results
Our analysis revealed a distinct "split personality" in ChatGPT's cultural profile.
Cognitive Style: A Holistic Bias
In cognitive process tests, both models showed a significant inclination toward Eastern holistic thinking. On the AHS, GPT-3.5 (M=5.35) and GPT-4 (M=5.01) scored significantly higher than the human norm (M=4.87, $p < 0.001$). In the TCT grouping task, the models chose relational (holistic) pairings 85-87% of the time, exceeding the mean of most Chinese provinces in comparison data.
Value Judgments: A Western Lean
Conversely, in value-based assessments, the models often leaned Western. On the SCS-Independent scale, GPT-3.5 scored a near-perfect 6.98/7.0, significantly higher than the human norm (4.73). While GPT-4 (5.32) was closer to human norms, it still leaned heavily toward Western individualism.
Robustness and Model Differences
The cultural profiles remained consistent across varying temperatures and prompt paraphrases, suggesting these are stable traits rather than stochastic "hallucinations." Notably, GPT-4 generally aligned more closely with human norms than GPT-3.5, likely due to more rigorous Reinforcement Learning from Human Feedback (RLHF) and alignment tuning.
Discussion
The holistic nature of ChatGPT's cognitive processing may be an inherent byproduct of the Transformer architecture, which predicts tokens based on the entire preceding context—a fundamentally holistic approach. However, the disconnect between process and value suggests that training data and RLHF act as cultural filters, potentially imposing Western values on a holistically-processing engine.
Future research should expand this inquiry to LLMs developed in different cultural contexts (e.g., Baidu’s Ernie Bot) to determine if holistic processing is universal to the Transformer architecture or specific to OpenAI’s development pipeline. Understanding these biases is essential for the responsible deployment of AI in diverse global markets.
Summary updated