Abstract
Detecting harmful memes in multicultural environments like Singapore presents unique challenges due to the complex interplay of visual metaphors and multilingual text. This paper introduces the OSPC framework, the winning solution for AI Singapore’s Online Safety Prize Challenge. The system employs a "catalyst" approach, distilling the reasoning capabilities of GPT-4V into a smaller, deployable Qwen-14B model. By integrating advanced image captioning, dual-model OCR, and targeted fine-tuning, OSPC significantly outperforms traditional multimodal benchmarks in identifying social biases across English, Chinese, Malay, and Tamil.
Introduction
Memes represent a significant challenge for AI safety because their harmful nature is often "nuanced and context-dependent," emerging from the interaction between image and text rather than either element in isolation. In multicultural hubs like Singapore, this complexity is compounded by the use of multiple languages and scripts. Existing detection systems, often trained on Western-centric datasets, struggle with low-resource languages like Tamil and the specific visual metaphors prevalent in local contexts.
Methodology
The OSPC framework utilizes a three-step pipeline designed for high accuracy and linguistic flexibility.
- Multimodal Feature Extraction: The system uses the BLIP model for image captioning. For text extraction, a dual-model OCR strategy is employed: PP-OCR handles standard text, while TrOCR serves as a high-confidence fallback for complex fonts or scripts like Tamil.
- Linguistic Bridging: To overcome the base LLM's limitations with Tamil, a dedicated Tamil-to-English Transformer translation model ensures the reasoning engine receives high-quality semantic input.
- LLM Reasoning (The "Catalyst"): The core classifier is a Qwen-1.5-14B model. To enhance its performance, the authors used GPT-4V to generate detailed reasoning annotations for the Hateful Memes Challenge Dataset. This "distilled" knowledge was then used to fine-tune Qwen via QLoRA.
Data Collection and Augmentation
The model was trained using a combination of established datasets and custom synthetic data:
- Hateful Memes Challenge Dataset: The primary base for fine-tuning.
- Synthetic OCR Dataset: To improve multilingual text recognition, the authors generated 52,000 synthetic images using ChatGPT for prompts and Stable Diffusion (SDXL) for image generation, overlaying them with multilingual text in various styles.
- Regional Corpora: Sourced from Malaysian and Singaporean internet data to train the Tamil translation model.
Results and Key Insights
The OSPC framework achieved first place in the Online Safety Prize Challenge, demonstrating substantial improvements over baseline models:
- AUROC: 0.7749 (vs. 0.5695 for the FLAVA baseline).
- Accuracy: 0.7087.
A key insight from this work is the effectiveness of the "Catalyst" strategy. By using GPT-4V as a teacher to explain why a meme is harmful, the researchers successfully transferred sophisticated reasoning logic to a much smaller, more efficient model. Additionally, providing the LLM with explicit definitions of harm categories (e.g., "Socio-Economic Divides") proved more effective than simple binary classification prompts.
Conclusion
While OSPC sets a new benchmark, challenges remain. The reliance on separate captioning models like BLIP can lead to information loss if the model fails to recognize specific cultural or historical figures. Furthermore, the "Tamil bottleneck" persists, as the performance of the student model is inherently limited by the teacher's (GPT-4V) own proficiency in low-resource languages. Future research will focus on "native" multimodal LLMs that can process visual and textual data simultaneously to minimize these gaps.
Summary updated