Text classification in fair competition law violations using deep learning

Yinying Kong, Niangqiu Huang, Haodong Deng, Junwen Feng, Xingyi Liang, Weisi Lv, Jingyi LiuFrontiers in Applied Mathematics and Statistics 2023 · DOI

TL;DR

Researchers have developed a deep learning-based approach to automate the review of government documents for fair competition law violations. By comparing Long Short-Term Memory (LSTM) and TextCNN models, the study demonstrates that these architectures—enhanced by data augmentation—can identify "suspect" documents with over 96% accuracy, significantly reducing the burden of manual legal review.

What Problem Does This Paper Address?

In many jurisdictions, including China, government agencies must review normative documents and policy measures to ensure they don't inadvertently create market monopolies or unfair advantages. Traditionally, this "fair competition review" is a grueling manual process. It requires legal experts to sift through massive volumes of formal, technical text to identify subtle violations.

The challenge lies in the nature of the data: government documents are lengthy, written in a highly formal style, and often lack the emotional cues found in social media text. Furthermore, instances of actual "violations" (suspect documents) are relatively rare compared to standard policy text, creating a "needle in a haystack" problem that is ripe for automation but difficult for standard machine learning models due to class imbalance.

Methods

The authors implemented and compared two distinct deep learning architectures to classify documents into three categories: Relevant, Suspect, and Irrelevant.

  1. LSTM (Long Short-Term Memory): A type of Recurrent Neural Network (RNN) designed to capture long-term dependencies in sequential data. This is particularly useful for legal text where the meaning of a sentence might depend on context established much earlier in the document.
  2. TextCNN (Convolutional Neural Network for Text): While CNNs are famous for image processing, TextCNN applies convolutional filters to word embeddings to capture local features and key patterns (like specific illegal phrasing) regardless of where they appear in the text.
  3. EDA (Easy Data Augmentation): To solve the scarcity of "suspect" examples, the authors used the NLPCDA tool to perform synonym replacement and random entity insertion. This expanded the "suspect" dataset from 1,512 to 4,536 instances.
  4. Preprocessing: The pipeline included the Jieba tool for Chinese word segmentation and the removal of stop words and numeric characters to focus the models on semantic content.

Datasets

The study utilized a proprietary dataset provided by the Antitrust Office of the Guangdong Province Market Supervision and Administration.

  • Composition: Exclusively Chinese government documents.
  • Classes:
    • Relevant: Pertains to market subjects but does not violate fair competition.
  • Suspect: Involves market subjects and potentially violates fair competition.
  • Irrelevant: Ordinary policy text not involving market players.
  • Split: 60% training, 20% validation, and 20% testing.

Key Insights and Ideas

  • Augmentation is the Great Equalizer: Before data augmentation, the LSTM outperformed TextCNN by nearly a full percentage point in accuracy. However, after applying EDA, both models converged at a high performance level (96.36% vs 96.19%), suggesting that data quality and volume are more critical than the specific choice between these two architectures.
  • Efficiency vs. Depth: While LSTM is slightly better at handling the sequential logic of long documents, it is computationally "expensive" because it processes text linearly. TextCNN, by contrast, can process text in parallel, making it three times faster to train (10 minutes vs. 30 minutes) while achieving nearly identical results.
  • Formal Language Suitability: The authors note that the lack of emotional bias and the structured nature of government documents actually make them excellent candidates for deep learning, as the models don't have to filter out the "noise" of slang or sarcasm found in other NLP tasks.

Key Results

The study compared the models across two scenarios:

Without Text Enhancement: * LSTM: 93.71% test accuracy. * TextCNN: 92.84% test accuracy.

With Text Enhancement (EDA): * LSTM: 96.36% test accuracy. * TextCNN: 96.19% test accuracy.

Beyond accuracy, the Recall rate (the ability to find all "suspect" documents) improved significantly with enhancement, reaching 92.45% for LSTM and 93.71% for TextCNN. In a regulatory context, high recall is vital because missing a violation is often worse than a false alarm.

Limitations and Future Work

While the results are promising, the authors acknowledge several areas for improvement:

  • Model Complexity: The study focused on LSTM and TextCNN. Future work could explore Transformer-based models (like BERT or RoBERTa), which have become the gold standard for NLP but require significantly more computational resources.
  • Dataset Scale: Although EDA helped, the base number of "suspect" files remains small. Expanding the raw dataset with more real-world examples from different provinces would improve the model's generalizability.
  • Interpretability: For legal applications, knowing why a model flagged a document as "suspect" is crucial. The current "black box" nature of these deep learning models may hinder their adoption by legal professionals who require explainable evidence.

Summary updated

Abstract

Introduction Ensuring fair competition through manual review is a complex undertaking. This paper introduces the utilization of Long Short-Term Memory (LSTM) neural networks and TextCNN to establish a text classifier for classifying and reviewing normative documents. Methods The experimental dataset used consists of policy measure samples provided by the antitrust division of the Guangdong Market Supervision Administration. We conduct a comparative analysis of the performance of LSTM and TextCNN classification models. Results In three classification experiments conducted without an enhanced experimental dataset, the LSTM classifier achieved an accuracy of 95.74%, while the TextCNN classifier achieved an accuracy of 92.7% on the test set. Conversely, in three classification experiments utilizing an enhanced experimental dataset, the LSTM classifier demonstrated an accuracy of 96.36%, and the TextCNN classifier achieved an accuracy of 96.19% on the test set. Discussion The experimental results highlight the effectiveness of LSTM and TextCNN in classifying and reviewing normative documents. The superior accuracy achieved with the enhanced experimental dataset underscores the potential of these models in real-world applications, particularly in tasks involving fair competition review.
BibTeX
@inproceedings{kong2023text,
  author = {Yinying Kong and Niangqiu Huang and Haodong Deng and Junwen Feng and Xingyi Liang and Weisi Lv and Jingyi Liu},
  title = {{Text classification in fair competition law violations using deep learning}},
  booktitle = {Frontiers in Applied Mathematics and Statistics},
  year = {2023},
  doi = {10.3389/fams.2023.1177081},
}
Figure