TL;DR
The authors propose OptIDC, a unified framework that jointly optimizes data selection at three levels: samples, features, and embedding units. By using a novel selection-weighted Optimal Transport mechanism to coordinate these multi-level decisions, OptIDC significantly improves Click-Through Rate (CTR) prediction accuracy across various model architectures while maintaining computational efficiency.
What Problem Does This Paper Address?
Deep CTR models, the backbone of modern recommendation systems and online advertising, are incredibly sensitive to the quality and configuration of their input data. Real-world training data is often "noisy" at multiple granularities:
- Sample Level: Some training examples are low-quality or irrelevant.
- Feature Level: Not all feature fields (e.g., "user age" vs. "item color") are equally predictive for every instance.
- Embedding-Unit Level: Within a feature's vector representation, some dimensions may be redundant or purely noise.
While existing research has tackled selection at each of these levels individually, they are typically optimized in isolation. This leads to selection conflicts: for example, a model might assign a high weight to a training sample but then mask out all the informative features within that sample. The authors argue that these three selection tasks are actually coupled decisions about the same effective input and need a unified, coordinated approach.
Methods
The core of the paper is the OptIDC framework, which integrates sample, feature, and embedding-unit selection into a single differentiable pipeline.
- Unified Learning Path: Selectors are arranged along the input path. Feature and embedding-unit selectors first "configure" the input tensor. Then, a sample selector evaluates this already-configured tensor to decide how strongly it should influence the model's learning.
- Selection-Weighted Optimal Transport (OT): This is the "secret sauce" for coordination. To ensure that coarse-grained decisions (like sample weighting) are supported by compatible fine-grained decisions (like feature selection), the authors formulate a transport problem. A coarse selection score is treated as "mass" that must be distributed among finer-grained "destinations" (features or units) based on their semantic compatibility. This penalizes the model if it tries to keep a sample while discarding its most relevant features.
- Two-Stage Optimization:
- Search Stage: The model learns continuous "gates" for selection using an annealing schedule that gradually pushes them toward binary (0 or 1) decisions.
- Retrain Stage: The learned configuration is "hardened" (fixed), and the backbone CTR model is retrained from scratch on the optimized input space.
Datasets
The framework was evaluated on four diverse, real-world benchmark datasets:
- Avazu: A large-scale advertising dataset.
- Criteo: A standard benchmark for display advertising.
- MovieLens-1M (ML1M): A classic movie recommendation dataset.
- AliCCP: A dataset from the Taobao e-commerce platform, reflecting real-world industrial scale.
Key Insights and Ideas
- The Common Masking Form: The authors realized that sample, feature, and unit selection can all be viewed as masks applied along different axes of the same input embedding tensor. This shared form is what makes a unified framework possible.
- Coordination is Crucial: Simply doing joint selection without explicit coordination (a baseline they call IDC) is better than single-level selection, but it still suffers from internal conflicts. The Optimal Transport mechanism provides a principled way to align decisions across different mathematical structures (scalars vs. vectors vs. matrices).
- Model Agnostic: OptIDC doesn't care which deep CTR model you use (DNN, DeepFM, DCN, etc.). It acts as a "pre-processor" that optimizes the information flow into the model.
Key Results
- Superior Performance: OptIDC consistently achieved higher AUC (Area Under the ROC Curve) than all single-level baselines and the uncoordinated joint baseline across all datasets and five different backbone architectures.
- Selection Consistency: Using a metric called the Selection Consistency Ratio (SCR), the authors showed that their OT coordination significantly reduces conflicts between levels, leading to more "agreeable" selection masks.
- Efficiency: While the search stage adds some training overhead, the final model has zero serving-time overhead for sample selection (since it's only used to filter the training set) and minimal overhead for feature/unit selection (implemented as simple lookup tables).
Limitations and Future Work
- Metric Discrepancy: On the AliCCP dataset, while OptIDC achieved the highest AUC, it did not always achieve the lowest LogLoss. The authors suggest that its benefits are more consistently reflected in ranking metrics (AUC) than in absolute probability calibration (LogLoss).
- Interaction Selection: The current work focuses on the input side. Future work could extend this unified framework to also select which feature interactions are most important, further compressing and optimizing the model.
- Hyperparameter Sensitivity: The framework introduces several new hyperparameters (like the OT penalty weight $\lambda_{co}$), which require careful tuning to balance sparsity and predictive performance.
Summary updated