TL;DR
This paper introduces the first large-scale, multimodal dataset of self-disclosure on Chinese social media and develops models to automatically detect it. They found that while users generally share moderate amounts of personal information, the type of disclosure varies significantly with their motivations, and multimodal models are crucial for understanding implicit self-disclosure.
What Problem Does This Paper Address?
Self-disclosure online offers psychological benefits but also poses privacy risks, especially as users increasingly share information across text and images. Existing research on self-disclosure detection has largely focused on text-only data in English, leaving a significant gap in understanding and addressing these risks in multimodal contexts and non-English languages, particularly Chinese. This paper aims to bridge this gap by creating a dedicated Chinese multimodal dataset and developing robust detection models.
Methods
The authors developed a novel annotation protocol for multimodal self-disclosure. This involved:
- LLM-based Pre-annotation: Using LLaVA-1.5 for image captioning and Qwen2.5-7B-Instruct for initial text-based annotation to generate candidate self-disclosure labels.
- Annotation Protocol Development: Creating a detailed document with definitions for personal information taxonomy, attribute value domains, and multimodal evidence integration rules.
- Expert Annotation: Employing six trained annotators using a custom tool that integrated image and text viewing for efficient labeling.
- Quality Control: Implementing cross-validation and the "Two Agree" method to ensure annotation reliability.
For detection, they framed the task as a question-answering problem and fine-tuned five multimodal large language models (MLLMs) and one text-based model (RoBERTa) on their newly created dataset.
Datasets
The paper introduces a new, high-quality Chinese multimodal self-disclosure corpus.
- Dataset Name: Chinese Multimodal Self-Disclosure Dataset
- Source: User-generated posts from the "Rednote" social media platform.
- Size: 4,870 multimodal (text and image) posts.
- Content: Posts were collected with a focus on "life-related attributes" and excluded commercial content.
- Annotation: Posts were annotated across five core categories and fifteen key personal information attributes, including personal identity, family/social relationships, health, occupation/education, and economic/financial status.
Key Insights and Ideas
- Moderate Disclosure is Common: The majority of posts (around 59%) contain only one to two self-disclosure attributes, suggesting users tend towards moderate information sharing.
- Motivation Drives Disclosure: Users' motivations significantly influence what they disclose. Posts driven by "Emotional Release and Resonance" or "Social Interaction and Belonging" tend to reveal more sensitive information (like gender, marital status) compared to posts focused on "Knowledge Sharing" or "Consumption Guidance."
- Multimodality is Key for Implicit Disclosure: For certain attributes, especially sexual orientation, inferring self-disclosure requires combining both text and image information. Text-only models struggle with these implicit cases.
- Chinese Context Matters: The study highlights the need for language-specific and culturally-aware models, as Chinese social media dynamics and semantics can differ from English contexts.
Key Results
- Model Performance: Fine-tuned MLLMs significantly outperformed non-fine-tuned models and the text-only RoBERTa model on self-disclosure detection.
- Top Performer: The Qwen2.5-omni-7B model achieved the highest partial span F1 score of 88.2% on the multimodal self-disclosure detection task.
- Implicit Disclosure: Qwen2.5-Omni-7B also demonstrated strong capabilities in handling implicit self-disclosure, achieving an F1 score of 75.00 for sexual orientation, far surpassing the text-only RoBERTa model which failed on this task.
- Data Imbalance: The dataset exhibits imbalanced distributions across different self-disclosure categories, which is noted as a limitation.
Limitations and Future Work
- Dataset Imbalance: The authors acknowledge that the dataset has imbalanced sample distributions, which could affect model generalization. Future work will focus on expanding the dataset with a greater emphasis on balancing attribute distributions.
- Implicit Disclosure Challenges: While MLLMs show promise, recognizing implicit self-disclosure still requires significant model capability and poses higher demands for safeguarding disclosed information.
- Cross-modal Reasoning: Despite advancements, there's still room for improvement in models' ability to accurately capture and reason across multimodal cues for self-disclosure detection.
Summary updated