Textual Content Moderation in C2C Marketplace

Yusuke Shido, Hsien-Chi Liu, Keisuke UmezawaACL 2022

TL;DR

This paper introduces a machine learning-based system for moderating user-generated messages in C2C marketplaces, significantly reducing human moderation costs. By leveraging conversational context and a human-in-the-loop approach for continuous improvement, their system achieves a substantial reduction in manual effort while maintaining high detection accuracy.

What Problem Does This Paper Address?

Online marketplaces, especially C2C platforms like Mercari, rely heavily on user-to-user communication to facilitate transactions. This communication, however, can be a breeding ground for various abuses, such as attempts to move transactions offline, solicitations for upfront payments via external services, or the transmission of offensive content. Traditional moderation methods, like keyword-based monitoring and manual patrols, struggle to scale effectively with the sheer volume of messages and the evolving nature of abusive tactics. This paper addresses the challenge of building an efficient, scalable, and accurate content moderation system for these dynamic C2C environments.

Methods

The authors propose a lightweight neural network that takes a conversation as input. Their system integrates three channels for detecting violations: random patrols, keyword/rule-based monitoring, and predictions from their machine learning (ML) models. A key aspect of their approach is a "human-in-the-loop" framework. Messages flagged as potential violations by the ML models are still manually reviewed by moderators. This not only ensures a high-quality, accurately labeled dataset for continuous model retraining but also prevents the negative customer experience that can arise from false positives.

The ML model itself is designed to be scalable and adaptable. It uses a one-vs-rest approach, where individual sub-language models are trained for different violation domains. The architecture involves:

  1. Textual Feature Extraction: A Convolutional Neural Network (CNN), specifically a KimCNN, is used to extract features from messages.
  2. Contextualization: The extracted message features are combined with "Writer" (seller/buyer) and "Status" (e.g., "waiting for payment") features.
  3. Sequential Understanding: Gated Recurrent Units (GRUs) process the sequence of messages (including their associated features) to understand the conversational context.
  4. Prediction: A Multi-Layer Perceptron (MLP) outputs the probability of a message violating terms and conditions.

The system is deployed as a microservice architecture for high availability and scalability.

Datasets

The authors utilize an in-house dataset collected from the Japanese version of the Mercari app. This dataset consists of conversations between sellers and buyers. Key features include:

  • Message: The text body of the message.
  • Writer: Indicates whether the message was sent by a seller or a buyer.
  • Status: The current status of the transaction (e.g., "waiting for payment," "waiting for shipping").

To comprehend the context, the model considers the most recent five messages in a conversation. The dataset was created by sampling positive and negative data with different ratios, and user personal identifiable information was masked during pre-processing. The data was split into training (10 months), validation, and testing sets (1 month each) to ensure robust training results.

Key Insights and Ideas

  • Conversational Context is Crucial: Incorporating multi-round messages and transaction status significantly improves the detection of violating messages compared to single-message analysis.
  • Human-in-the-Loop for Continuous Improvement: The manual review of flagged messages is not just a safety net but a vital component for generating high-quality training data, enabling the system to adapt to evolving violation types.
  • Scalability through One-vs-Rest: Serving violation domains as individual components in a one-vs-rest fashion allows for flexible scaling and easier addition of new moderation models.
  • ML Outperforms Rules in Efficiency: While rule-based systems are preserved for rapid adaptation to new violations, ML models demonstrate a significant reduction in the number of alerts needed to achieve comparable detection rates.

Key Results

The proposed ML-based system successfully reduced human operation costs to less than one-sixth compared to the conventional rule-based monitoring at Mercari. In online evaluation, the model was able to find the same number of positive messages as existing rule-based methods while generating only 16.05% of the alerts. This implies a theoretical reduction of human resources by more than half. The paper also reports high AUROC and AUPRC scores, with the best performing model achieving 99.7274 AUROC and 98.4194 AUPRC when using multi-round messages and writer/status features.

Limitations and Future Work

The authors acknowledge that while their ML models achieve high precision, the dynamic nature of marketplace violations means that updating ML-based methods to keep pace with evolving tactics remains a challenge. They preserve rule-based approaches to quickly accommodate new types of violations. Future work could focus on further improving the ML models' ability to adapt to concept drift and exploring more sophisticated methods for understanding nuanced abusive intents. The paper also mentions that the system was trained on the Japanese version of the Mercari app, suggesting potential for cross-lingual adaptation or further investigation into language-specific nuances.

Summary updated

BibTeX
@inproceedings{shido2022textual,
  author = {Yusuke Shido and Hsien-Chi Liu and Keisuke Umezawa},
  title = {{Textual Content Moderation in C2C Marketplace}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure