TL;DR
This paper introduces PU GNN, a novel method for detecting chargeback fraud in Play-to-Earn (P2E) MMORPGs. By combining player behavior logs with transaction graph data and employing a Positive-Unlabeled (PU) learning approach to handle imbalanced and partially labeled data, PU GNN significantly outperforms existing methods.
What Problem Does This Paper Address?
The rise of Play-to-Earn (P2E) Massively Multiplayer Online Role-Playing Games (MMORPGs) has blurred the lines between virtual in-game assets and real-world value. Players can now exchange in-game goods for cryptocurrencies, making these assets directly convertible to real-world currency. However, this also introduces a significant challenge: chargeback fraud. Unlike traditional in-game items, P2E assets recorded on the blockchain are difficult to recover once fraudulent transactions occur (e.g., payment fraud, cancellations, or refunds). Existing fraud detection methods often struggle with the unique characteristics of P2E MMORPGs, particularly the imbalanced nature of fraud data and the difficulty in obtaining definitive labels for all players, as some players who haven't yet committed fraud might do so later. This paper aims to address this gap by developing a robust chargeback fraud detection system specifically for P2E MMORPGs.
Methods
The proposed PU GNN model integrates two key data sources: player in-game activity logs and P2E token transaction graphs. The architecture consists of four main components:
- Player Behavior Modeling: This layer processes players' sequential in-game activity logs (e.g., logins, item purchases, quest completions) using a self-attention mechanism. This captures nuanced behavioral patterns that might indicate fraudulent activity. The output is an initial behavioral representation for each player.
- Graph Attention Networks (GATv2): The P2E token transaction graph, where nodes represent players and edges represent token transfers, is utilized. GATv2 is employed to learn contextual relationships between players by aggregating information from their neighbors in the transaction network. This component captures how players interact and transact, which can reveal collusive or fraudulent behavior.
- Imbalanced Label Handling (GraphSMOTE): To address the severe imbalance between fraudulent and benign players, the authors adapt GraphSMOTE. This technique oversamples the minority class (fraudulent players) by generating synthetic graph data, helping to balance the dataset.
- Positive & Unlabeled Learning (PU Loss): A crucial aspect of the method is its handling of unlabeled data. In the context of chargeback fraud, players who have not yet committed fraud but could potentially do so are treated as unlabeled. The PU learning framework, specifically using a non-negative PU (nnPU) risk estimator, allows the model to learn from both confirmed fraud (positive) and unlabeled instances, rather than treating all unlabeled instances as definitively benign. This is essential because players might cancel payments after mining P2E tokens.
The model combines the player behavior representations and the graph-based representations, then uses a PU loss function to train the classifier.
Datasets
The authors evaluated their model on three real-world datasets collected from two popular P2E MMORPGs. These datasets consist of player behavior activity logs and P2E token transaction graphs. The game and token titles are anonymized as MMORPG01, MMORPG02, TOKEN01, TOKEN02, and TOKEN03. The datasets were split into training, validation, and test sets. For the PU learning setup, half of the positive (fraud) labels were treated as positive, and the other half were treated as unlabeled.
Key Insights and Ideas
- The Value of Integrated Data: Combining player behavior logs with transaction graph data is crucial for effective chargeback fraud detection in P2E MMORPGs. Neither source alone is sufficient; their synergy provides a more comprehensive view of player activities.
- Unlabeled Data is Not Necessarily Benign: In the P2E context, players who haven't yet committed fraud but have paid for in-game goods are best treated as "unlabeled" rather than definitively "benign." This is because they might still initiate a chargeback after acquiring P2E tokens. PU learning effectively handles this uncertainty.
- GraphSMOTE for Imbalanced Graphs: Adapting SMOTE for graph-structured data (GraphSMOTE) is effective in mitigating the severe class imbalance common in fraud detection tasks, leading to better model performance.
- Automated Fraud Patterns: Chargeback fraud in P2E MMORPGs often stems from automated programs (macros) rather than individual manual play, leading to distinct, collusive activity patterns that can be detected.
Key Results
The PU GNN model demonstrated superior performance across all three datasets compared to various baseline models, including MLP, GRU, GCN, GraphSAGE, and GAT.
- F1-score and AUC: PU GNN achieved the best F1-scores and AUC values on all datasets. For instance, on DATASET01, PU GNN achieved an F1-score of 0.855 and an AUC of 0.854, outperforming the next best baseline (DAGNN) which scored 0.846 and 0.835 respectively.
- Sensitivity and Specificity: The model also showed strong performance in terms of sensitivity (true positive rate) and specificity (true negative rate), indicating its ability to correctly identify both fraudulent and benign players.
- Ablation Studies: Ablation studies confirmed the effectiveness of both the PU learning approach and the GraphSMOTE oversampling technique. Models trained without these components performed significantly worse.
Limitations and Future Work
The authors acknowledge several areas for improvement:
- Temporal Features: The current model does not explicitly consider time-related temporal features, such as the time interval between payment and cancellation, or collusive work times. Incorporating survival analysis techniques like hazard functions could enhance detection.
- Heterogeneous Graphs: The current graph representation is homogeneous. Extending the model to handle heterogeneous graphs with various node and edge types could capture more complex relationships.
- Early Detection: Developing methods for earlier detection of chargeback fraud is crucial for minimizing losses.
- Unlabeled Data Interpretation: While PU learning is effective, further research into the precise interpretation and utilization of unlabeled data could yield even better results.
Summary updated