Detecting Transaction-based Tax Evasion Activities on Social Media Platforms Using Multi-modal Deep Neural Networks

Lelin Zhang, Xi Nan, Eva Huang, Sidong LiuarXiv 2020 · OpenReview

Abstract

Social media platforms now serve billions of users by providing convenient means of communication, content sharing and even payment between different users. Due to such convenient and anarchic nature, they have also been used rampantly to promote and conduct business activities between unregistered market participants without paying taxes. Tax authorities worldwide face difficulties in regulating these hidden economy activities by traditional regulatory means. This paper presents a machine learning based Regtech tool for international tax authorities to detect transaction-based tax evasion activities on social media platforms. To build such a tool, we collected a dataset of 58,660 Instagram posts and manually labelled 2,081 sampled posts with multiple properties related to transaction-based tax evasion activities. Based on the dataset, we developed a multi-modal deep neural network to automatically detect suspicious posts. The proposed model combines comments, hashtags and image modalities to produce the final output. As shown by our experiments, the combined model achieved an AUC of 0.808 and F1 score of 0.762, outperforming any single modality models. This tool could help tax authorities to identify audit targets in an efficient and effective manner, and combat social e-commerce tax evasion in scale.

Introduction

The "hidden economy"—transactions that go unrecorded to avoid tax liability—represents a multi-billion dollar challenge for governments globally. While traditional auditing focuses on registered businesses, social media has enabled a massive, decentralized marketplace of unregistered participants. A prime example is the "Daigou" (purchasing agent) phenomenon, where individuals buy luxury goods or popular products abroad and sell them via social media to avoid import duties and income tax. Because these transactions are often concealed within personal posts and use informal payment methods, they are incredibly difficult for authorities like the Australian Taxation Office (ATO) to track without labor-intensive manual searching.

Methodology

The researchers propose a Multi-modal Deep Neural Network (DNN) architecture designed to process the diverse data types found in a typical Instagram post. The model consists of three primary branches:

  1. Text Modality (Hashtags & Comments): Two separate adapter-BERT models are utilized. One processes the hashtags, while the other processes the comments. BERT (Bidirectional Encoder Representations from Transformers) provides deep contextual understanding, and the "adapter" architecture allows for parameter-efficient fine-tuning on social media slang and informal language.
  2. Image Modality: For visual content, the system employs EfficientNetB7, a state-of-the-art convolutional neural network known for its high accuracy and efficiency in object detection and classification.
  3. Feature Fusion: The features from these three branches (768 dimensions from each BERT model and 2,560 from EfficientNet) are concatenated into a single 4,096-dimensional vector. This joint representation is then passed through a dropout layer and a dense output layer to classify the post as "tax evasion" or "not."
The proposed multi-modal architecture combining image, hashtag, and comment data streams.

Experimental Setup

The researchers created a custom dataset specifically for this task, focusing on Instagram posts tagged with #lipstick, identified as a high-volume, high-risk category for informal sales. From an initial crawl of 58,660 posts, a subset of 2,081 posts was manually labeled by three experts across nine properties, including "selling intention," "source of the post" (e.g., brand vs. unregistered producer), and "relation to hidden economy."

Results and Discussion

The experimental results highlight the significant power of multi-modality. The most critical insight is the complementary nature of text and images. Models relying solely on hashtags achieved high recall but suffered from low precision (many false positives). Conversely, image-only models demonstrated higher precision but struggled with recall. By fusing these modalities, the model achieved a balanced and superior performance.

Precision-Recall curves demonstrating that the multi-modal model (AUC 0.808) outperforms all single-modality baselines.

As shown in the performance metrics, the multi-modal approach significantly outperformed single-modality models:

  • Multi-modal: AUC = 0.808, F1 = 0.762
  • Hashtags only: F1 = 0.593
  • Comments only: F1 = 0.742
  • Images only: F1 = 0.696

From a practical standpoint, the authors calculate that while a random sample might yield 22 tax evasion cases per 100 posts, their model identifies "suspicious" targets such that an officer would find 72 cases per 100 reviews—effectively tripling the efficiency of human auditors.

Conclusion and Future Work

This study demonstrates that multi-modal deep learning can effectively identify unregistered commercial activity hidden within social media platforms. While the current model shows high efficacy, future work will address the "video gap" (incorporating video analysis) and expand the product scope beyond cosmetics to include electronics and designer fashion. Additionally, integrating user metadata—such as follower counts and historical posting behavior—could further enhance the model's ability to distinguish between personal accounts and shadow businesses.

Summary updated

Abstract

Social media platforms now serve billions of users by providing convenient means of communication, content sharing and even payment between different users. Due to such convenient and anarchic nature, they have also been used rampantly to promote and conduct business activities between unregistered market participants without paying taxes. Tax authorities worldwide face difficulties in regulating these hidden economy activities by traditional regulatory means. This paper presents a machine learning based Regtech tool for international tax authorities to detect transaction-based tax evasion activities on social media platforms. To build such a tool, we collected a dataset of 58,660 Instagram posts and manually labelled 2,081 sampled posts with multiple properties related to transaction-based tax evasion activities. Based on the dataset, we developed a multi-modal deep neural network to automatically detect suspicious posts. The proposed model combines comments, hashtags and image modalities to produce the final output. As shown by our experiments, the combined model achieved an AUC of 0.808 and F1 score of 0.762, outperforming any single modality models. This tool could help tax authorities to identify audit targets in an efficient and effective manner, and combat social e-commerce tax evasion in scale.
BibTeX
@misc{zhang2020detecting,
  author = {Lelin Zhang and Xi Nan and Eva Huang and Sidong Liu},
  title = {{Detecting Transaction-based Tax Evasion Activities on Social Media Platforms Using Multi-modal Deep Neural Networks}},
  year = {2020},
  eprint = {2007.13525},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2007.13525},
}
Figure