TL;DR
This paper explores how social media users perceive and interact with automatic machine translation (MT) of their posts. It reveals that users are concerned about the accuracy and potential misinterpretations of MT, especially for sensitive or nuanced content, and desire more control over how their posts are translated and who sees them.
What Problem Does This Paper Address?
Social networking sites (SNS) increasingly offer machine translation (MT) to bridge language barriers and foster connectivity. However, these MT systems are not perfect and can lead to misunderstandings, potentially harming users' personal or professional relationships. A significant issue is that authors of posts often have no control over how their content is translated, nor can they view or correct these translations. This lack of control leaves authors vulnerable to misrepresentation and can impact their posting behavior. This paper investigates users' awareness, appreciation, and desire for control over MT of their social media posts to address this gap.
Methods
The researchers employed a mixed-methods approach, combining a survey study with semi-structured interviews.
- Survey Study: A web-based survey was conducted with 189 participants who met specific criteria for being bi/multilingual. The survey comprised 39 items covering demographic information, SNS and language preferences, awareness and experience with MT quality, perceived sensitivity of posts, and preferences for MT control. Many questions were adapted from existing surveys.
- Interview Study: Fifteen participants, recruited from the survey pool and through SNS posts, underwent semi-structured interviews. These interviews aimed to delve deeper into users' language use on SNS, their thoughts on MT quality, how they decide which posts are acceptable for translation, and what additional controls they would desire.
Data analysis involved quantitative methods (correlation, regression, t-tests) for survey data and thematic analysis for interview data.
Datasets
The paper does not explicitly mention or introduce new datasets for training MT models. The data used for analysis comes from the survey responses (N=189) and interview transcripts (N=15) collected from participants.
Key Insights and Ideas
- Awareness doesn't always mean acceptance: While a high percentage of users are familiar with MT (85%) and have encountered it on SNS (80%), this doesn't automatically translate to full trust or satisfaction.
- Nuance and slang are MT's Achilles' heel: Users are particularly concerned about MT's ability to accurately translate slang, idioms, and connotative meanings. Mistranslations in these areas can significantly distort the intended message, leading to confusion or even offense.
- Control is paramount for sensitive content: Users want to control which of their posts are translated, especially those discussing sensitive topics like politics, religion, or personal issues. They often use specific languages to target particular audiences and fear that MT could expose this content to unintended readers.
- Intentional language choice: Users deliberately choose languages for their posts to reach specific audiences. MT that indiscriminately translates these posts can undermine this intentionality and reach unintended readers.
- Desire for granular control: Users expressed a desire for features that would allow them to:
- Edit translations: Correcting inaccurate MT to ensure their message is conveyed precisely.
- Filter audiences: Selecting who can see the translated version of their posts, aligning with their original language choice.
Key Results
- Familiarity and Posting Frequency: Users more familiar with MT tend to post more frequently on SNS.
- Perceived MT Quality: A majority of participants generally found MT quality to be good. However, when examining the accuracy of actual meaning, those who post more frequently found MT to be more accurate.
- Experience vs. Accuracy: Users with prior MT experience perceived MT as less accurate in conveying connotative meaning, suggesting that direct experience highlights its limitations.
- Native Language Use: Participants who find MT more accurate in connotative meaning tend to post more often in their native language.
- Audience Targeting: Users who write posts in languages other than English are less likely to want their posts translated every time, often because they are intentionally targeting specific audiences.
- Qualitative Findings: Interviewees shared specific examples of mistranslations of slang and idioms, highlighting how MT can distort tone and meaning, leading to concerns about professional and personal repercussions.
Limitations and Future Work
- Participant Demographics: The study primarily recruited Spanish speakers and writers residing in the US. This linguistic diversity might limit the generalizability of findings regarding MT quality and accuracy, especially for low-resourced languages. Future work should include a more linguistically diverse sample.
- Controlled Experimentation: While the study advocates for control features, a controlled experiment is needed to confirm their impact on user comfort, satisfaction, and perceived control.
- Prototype Development: The authors plan to develop and test prototypes of translation features, including options to read but not edit, add manual translations, and edit MT. This will help measure the effectiveness of different control mechanisms.
- Ethical Considerations: The authors acknowledge that allowing users to edit translations could raise ethical concerns, such as the potential for creating language-specific misinformation. Presenting both the original and edited versions could be a solution.
Summary updated