Interaction-Aware Factorization Machines for Recommender Systems

Fuxing Hong, Dongbo Huang, Ge ChenAAAI 2019 · DOI

Introduction

In the domain of recommender systems, the ability to model complex feature interactions—such as how a user’s location might influence their app choice differently depending on the time of day—is fundamental to predictive accuracy. While Factorization Machines (FMs) have long been a standard tool for this task, they typically suffer from a "fairness" problem: they treat every feature interaction as equally important. This uniform weighting often introduces noise from irrelevant features and fails to prioritize highly informative relationships.

To address this, researchers from Tencent introduced the Interaction-aware Factorization Machine (IFM) at AAAI-19. The IFM employs an Interaction-Aware Mechanism (IAM) to learn which interactions are most significant, effectively prioritizing signal over noise.

The Problem: Uniform Interaction Weights

Standard FMs calculate the relationship between every pair of features in a dataset. This approach has two major flaws:

  1. Feature-level Noise: Not all features are equally useful. Including interactions from irrelevant features can degrade model accuracy. Furthermore, a feature's importance often depends on the specific feature it is paired with.
  2. Field-level Disparity: Features belong to broader categories or "fields" (e.g., "Gender," "Location"). Certain field combinations are inherently more informative than others. For instance, the interaction between "Phone Brand" and "Location" might be a much stronger signal for a recommendation than "Gender" and "Location."

Previous attempts to solve this either used simple attention mechanisms that ignored field structures or utilized field-aware latent vectors, which significantly increased parameter complexity.

Methods: The Interaction-Aware Mechanism (IAM)

The core innovation of the IFM is the Interaction-Aware Mechanism (IAM), which scales each pairwise interaction by a learned weight. This weight is derived from two distinct perspectives:

  • Feature Aspect (Attention): The model uses an attention network to evaluate specific feature pairs and determine the appropriate level of attention for their interaction.
  • Field Aspect (Prototypes): To capture category-level importance, the authors learn a "field-aware factor importance vector." To maintain computational efficiency, tensor factorization is used to decompose this importance matrix into a compact, low-dimensional representation.
The architecture of the Interaction-aware Factorization Machine (IFM), illustrating the flow from the Input Layer through the Interaction-Aware Mechanism (Attention Net and Projection Matrix) to the final output.

The authors also proposed two neural variants: the Interaction-aware Neural Network (INN), which captures complex non-linear relationships, and DeepIFM, a hybrid model combining the strengths of IFM and standard Deep Neural Networks. To handle massive datasets, a dynamic sampling scheme selects the most important interactions during training, significantly reducing computation time.

Experimental Results

The models were evaluated on two widely-used benchmarks: MovieLens (personalized tag recommendation) and Frappe (context-aware app recommendation).

Performance Comparison

The IFM and its neural counterpart (INN) consistently outperformed state-of-the-art baselines such as DeepFM, NFM, and AFM. On the MovieLens dataset, the INN model achieved an RMSE improvement of approximately 10% compared to the standard Factorization Machine.

RMSE performance of IFM versus standard FM across different keep probabilities on the MovieLens and Frappe datasets, demonstrating IFM's superior accuracy and robustness.

Key Insights

  • Hierarchical Understanding: By splitting importance into feature and field levels, the model gains a stratified understanding of data. It learns specific user-item preferences while also recognizing broader field-level patterns (e.g., that Movie-Tag interactions are generally more critical than User-Movie interactions in the MovieLens dataset).
  • Efficiency and Scalability: In real-world tests on Tencent video advertising data involving 10 billion instances, the sampling scheme reduced training time from 16 hours to under 4 hours with a negligible impact on accuracy.

Conclusion and Future Work

The Interaction-aware Factorization Machine provides a robust framework for prioritizing feature interactions in recommender systems. By leveraging both attention and field-aware prototypes, it achieves high expressiveness without the parameter explosion typical of previous field-aware models. Future research may explore the use of Neural Architecture Search (NAS) to automatically discover even more flexible interaction structures.

Summary updated

BibTeX
@inproceedings{hong2019interaction,
  author = {Fuxing Hong and Dongbo Huang and Ge Chen},
  title = {{Interaction-Aware Factorization Machines for Recommender Systems}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33013804},
}
Figure