Abstract
E-commerce companies have to face abnormal sellers who sell potentially-risky products. Typically, the risk can be identified by jointly considering product content (e.g., title and image) and seller behavior. This work focuses on behavior feature extraction as behavior sequences can provide valuable clues for the risk discovery by reflecting the sellers' operation habits. Traditional feature extraction techniques heavily depend on domain experts and adapt poorly to new tasks. In this paper, we propose a self-supervised method InfoBehavior to automatically extract meaningful representations from ultra-long raw behavior sequences instead of the costly feature selection procedure. InfoBehavior utilizes Bidirectional Transformer as feature encoder due to its excellent capability in modeling long-term dependency. However, it is intractable for commodity GPUs because the time and memory required by Transformer grow quadratically with the increase of sequence length. Thus, we propose a hierarchical grouping strategy to aggregate ultra-long raw behavior sequences to length-processable high-level embedding sequences. Moreover, we introduce two types of pretext tasks. Sequence-related pretext task defines a contrastive-based training objective to correctly select the masked-out coarse-grained/fine-grained behavior sequences against other "distractor" behavior sequences; Domain-related pretext task designs a classification training objective to correctly predict the domain-specific statistical results of anomalous behavior. We show that behavior representations from the pre-trained InfoBehavior can be directly used or integrated with features from other side information to support a wide range of downstream tasks. Experimental results demonstrate that InfoBehavior significantly improves the performance of Product Risk Management and Intellectual Property Protection.
Introduction
In the cat-and-mouse game of e-commerce risk management, "abnormal" sellers are experts at bypassing content-based detectors. They can easily tweak product titles or images to avoid being flagged. However, their behavioral patterns—how they log in, how frequently they edit products, and their operational habits—are much harder and more expensive to fake. These operational habits are "high-cost" signals that provide stable clues for anomaly detection.
The primary challenge is that these behavior sequences are "ultra-long." A single active seller can generate millions of actions over a month. Standard deep learning architectures like Transformers have a "quadratic memory problem," making it computationally impossible to process raw, month-long sequences on standard hardware. Traditionally, this required manual feature engineering, which is labor-intensive and often fails to capture complex temporal dependencies.
The InfoBehavior Framework
InfoBehavior is a self-supervised framework designed to learn representations from these massive sequences by combining a novel data-processing pipeline with Transformer-based encoding.
Methodology
The framework addresses the sequence length challenge through a Hierarchical Grouping Strategy:
- Minute-level Aggregation: Actions within the same minute are aggregated. The system uses "fuzzy counts" (Low, Medium, High) for repetitive actions and includes metadata like geographical location and time lags.
- CNN-based Aggregation: A sliding-window Convolutional Neural Network (CNN) further aggregates these minute-level representations into "span representations." This reduces the sequence length from tens of thousands to just a few hundred tokens, making them processable by a Transformer.
Self-Supervised Pre-training
Since labeled data is scarce, InfoBehavior learns from unlabeled raw logs using three pretext tasks:
- Coarse-grained Prediction: A contrastive task that asks the model to recognize if two different month-long sequences belong to the same seller.
- Fine-grained Prediction: A masked-modeling task where the model must predict a hidden behavior fragment based on its surrounding context.
- Domain-related Task: A classification task where the model predicts domain-specific statistics, such as failed login or verification attempts, which are strong indicators of risk.
Experimental Evaluation
The study utilized massive, real-world datasets from Alibaba Group, pre-training on behavior sequences from over 9.8 million sellers.
- Offline Performance: InfoBehavior achieved an AUC of 0.9171 on abnormal seller detection, significantly outperforming models using manually crafted features (0.9099).
- Online Impact: In real-world deployment, it improved recall by 3.86% for Product Risk Management (e.g., gambling aids) and by 4% to 5% for Intellectual Property Protection (e.g., counterfeits) across various categories.
Conclusion
InfoBehavior demonstrates that hierarchical grouping is a principled way to preserve spatial and temporal attributes while making ultra-long data "Transformer-ready." Ablation studies highlighted that the coarse-grained task (identifying overall seller habits) was most critical, suggesting that a seller's general "style" is more indicative of risk than isolated actions. The model is currently deployed at scale, protecting millions of consumers daily.
Summary updated