TL;DR
This paper tackles the challenge of automatically detecting censored tweets in Chinese by fine-tuning the XLNet language model. They achieved competitive results, demonstrating XLNet's effectiveness in this nuanced censorship detection task.
What Problem Does This Paper Address?
In today's digital age, social media platforms are crucial for information dissemination. However, censorship, whether by governments or private organizations, can suppress public discourse on sensitive or objectionable topics. Identifying and classifying censored content, especially in languages with complex character sets like Chinese, is a significant challenge. This paper aims to address this by developing a machine learning model to distinguish between censored and uncensored Chinese tweets.
Methods
The authors employed a supervised learning approach for binary classification (censored vs. uncensored). They experimented with several pre-trained transformer-based language models, including BERT, DeBERTa, ELECTRA, and XLNet. The core of their methodology involved fine-tuning these models on a custom dataset of Chinese tweets. Specifically, they utilized XLNet, a generalized autoregressive pre-training method, for its ability to capture long-range dependencies and its permutation language modeling objective. The process involved data preprocessing, tokenization using XLNetTokenizer, and then fine-tuning the XLNet model for sequence classification.
Datasets
The dataset used in this study was collected via a web scraper over a four-month period and contains Chinese tweets labeled as either censored ('1') or uncensored ('0'). The dataset was split into training, validation, and test sets. The authors do not mention introducing a new benchmark dataset but rather utilize an existing one for the NLP4IF 2021 workshop.
Key Insights and Ideas
- XLNet's Efficacy: XLNet demonstrated superior performance compared to other transformer models like BERT, DeBERTa, and ELECTRA for this specific censorship detection task. This suggests that its autoregressive nature and permutation language modeling approach are well-suited for understanding the subtle linguistic cues in censored Chinese tweets.
- Importance of Preprocessing: The paper highlights the necessity of data preprocessing, including filtering noisy information, to improve model performance.
- Fine-tuning for Domain Specificity: Fine-tuning pre-trained models on domain-specific data (censored tweets) is crucial for achieving accurate classification.
Key Results
The paper reports that XLNet achieved a validation accuracy of 0.634, outperforming other models. On the test set, XLNet achieved a precision of 0.65 and a recall of 0.64. The majority class baseline accuracy was 49.98%, and a human baseline accuracy was 23.83%, indicating that the XLNet model significantly surpassed these baselines.
Limitations and Future Work
The authors acknowledge that their work focuses on binary classification and that censorship can be a more complex phenomenon. They suggest that future work could involve exploring other NLP models and extending the censorship detection capabilities to other languages. Additionally, further investigation into more nuanced censorship detection methods beyond simple binary classification would be beneficial.
Summary updated