TL;DR
This study investigates how quadrilingual speakers in Aruba and Curaçao navigate interactions with Large Language Models (LLMs) across four languages: English, Dutch, Spanish, and the low-resource Creole Papiamentu/o. The researchers found that while English remains the "default" for AI interaction, users naturally switch languages based on the conversation topic—moving toward local languages for cultural discussions—and generally feel as comfortable switching with a chatbot as they do with a human.
What Problem Does This Paper Address?
Most research on code-switching—the practice of alternating between two or more languages in a single conversation—focuses on bilingual pairs of high-resource languages (like English-Spanish). However, millions of people live in truly multilingual environments where they juggle three, four, or more languages daily.
The authors tackle a critical gap: how do these speakers interact with AI agents that support their languages unequally? In the Caribbean islands of Aruba and Curaçao, residents are predominantly quadrilingual. While English and Spanish are "high-resource" (well-supported by AI), Dutch is mid-resource, and Papiamentu/o is a "low-resource" Creole that has only recently been added to major translation systems. The paper explores whether users "narrow" their linguistic repertoire to accommodate the AI or if they bring their full, fluid multilingual identity to the chat.
Methods
The researchers conducted a preregistered interaction study with 42 university students from Aruba and Curaçao, plus a comparison group of 20 international students.
The study procedure followed a structured workflow:
- Language Selection: Participants chose a preferred language for the survey interface.
- Chat 1 (Institutional): A 5-minute conversation about university life and career goals.
- Survey 1: Assessing AI familiarity and attitudes.
- Chat 2 (Cultural): A 5-minute conversation about local food, traditions, and culture.
- Survey 2: Detailed language background and switching comfort.
To analyze the results, the team used an LLM-as-a-judge approach. They employed GPT-5.4-mini to identify the dominant language of every message and classify the "triggers" that caused a user to switch languages.
Datasets
The study did not use a pre-existing corpus for training but instead generated a novel dataset of interaction logs:
- Primary Sample: 42 local students (predominantly quadrilingual).
- Comparison Sample: 20 international students (mostly English-dominant).
- Models Used: Participants interacted with either GPT-5.5 or Llama 4 Maverick. The system prompts explicitly instructed the models to detect the user's language and respond in kind to ensure the AI didn't "force" a specific language choice after the initial English greeting.
Key Insights and Ideas
- Situational vs. Metaphorical Switching: Users engaged primarily in situational switching. They didn't just switch randomly; they moved toward Papiamentu/o and Spanish when the topic shifted to local culture, mirroring real-world social norms where Dutch/English are for "work" and Papiamentu/o is for "home."
- The "English Default" Bias: Despite being quadrilingual, participants used English for the majority of messages. This suggests a "technological unmarked code"—an assumption that AI "works better" in English, likely learned from previous experiences with less capable systems.
- Turn-by-Turn over Intra-sentential: Unlike spoken conversation where speakers might mix languages mid-sentence, these users almost exclusively switched between turns. This reflects a common pattern in digital text-based communication where the "message" is the unit of thought.
- The Familiarity Paradox: Interestingly, participants who reported higher familiarity with AI switched languages less often. This suggests that "power users" have learned to accommodate the machine's perceived linguistic preferences to avoid errors.
Key Results
- Topic-Driven Shifts: In the GPT group, the use of Papiamentu/o and Spanish jumped from a mere 2.8% in the university chat to 22.4% in the cultural chat.
- English Dominance: English accounted for roughly 50% to 80% of all user messages across both topics, even though it is rarely the primary language spoken at home for these participants.
- Switching Triggers: The vast majority of switches (89–97%) occurred when the AI asked a new question, indicating that users see a new question as an "opening" to reset the linguistic context.
- Comfort Levels: There was no statistically significant difference in reported comfort when switching languages with an AI versus a human. Users are ready for multilingual AI, even if the AI isn't always ready for them.
Limitations and Future Work
The authors acknowledge several constraints:
- The "English Opening": The chatbot always started in English. This likely anchored the conversation and may have suppressed more frequent switching.
- Sample Size and Diversity: The sample was relatively small and concentrated in specific academic fields.
- Model Confounding: Participants were not randomly assigned to GPT or Llama models, making it difficult to draw causal conclusions about model performance.
- Future Directions: The researchers suggest that future AI evaluations must move beyond testing languages in isolation. Systems need to be tested on their ability to follow a user across language boundaries in a single session, especially when moving into low-resource languages.
Summary updated