Abstract
In collaborative tagging systems, recommending relevant tags for user-item pairs involves modeling complex ternary relationships. While linear tensor factorization methods like Pairwise Interaction Tensor Factorization (PITF) offer scalability, they often require a large number of latent features to achieve high accuracy. This paper presents Nonlinear Tensor Factorization (NLTF), a novel approach that replaces traditional linear dot products with Gaussian radial basis functions (RBF). By treating latent interactions as distances rather than projections, NLTF enables more expressive modeling within a lower-dimensional space. Our evaluation on large-scale datasets, including a 0.6 billion triple dataset, demonstrates that NLTF significantly outperforms linear baselines in both accuracy and feature efficiency while maintaining linear-time complexity.
Introduction
Collaborative tagging systems, such as Last.fm and Delicious, allow users to annotate items with descriptive tags. Recommending these tags is a "three-way" problem involving users, items, and tags, typically addressed through Tensor Factorization. The primary challenge in this domain is balancing recommendation accuracy with computational efficiency.
Traditional models face a significant trade-off. Tucker Decomposition (TD) provides high accuracy but suffers from $O(R^3)$ complexity, making it impractical for large-scale systems. Conversely, Canonical Decomposition (CD) models like PITF scale linearly ($O(R)$) but rely on linear dot products, which act as simple two-class classifiers. This limitation necessitates a high number of latent features to capture multi-faceted relationships, which can be memory-intensive and prone to overfitting.
Methodology
NLTF extends the CD approach by introducing a nonlinear Gaussian kernel. Instead of a standard dot product ($u \cdot t$), NLTF computes similarity using the Gaussian radial basis function (RBF): $\exp(-\frac{1}{2}\beta||p-q||^2)$. This shift effectively measures the Euclidean distance between latent feature vectors rather than their projection.
Following the PITF framework, NLTF decomposes the ternary user-item-tag relation into pairwise user-tag and item-tag interactions. The model can be interpreted as a three-layer neural network where the hidden layer utilizes RBF activation functions. Training is conducted via Stochastic Gradient Descent (SGD) using a pairwise ranking loss function, optimizing the model to rank observed tags higher than unobserved ones for any given user-item pair.
Key Insights: From Linear to Nonlinear
The core advantage of NLTF lies in its transition from "two-class" to "multi-class" classification. A linear dot product can only distinguish between two sides of a hyperplane (e.g., a feature is either "comedy" or "not comedy"). In contrast, the Gaussian kernel allows a single latent feature dimension to represent multiple distinct categories by positioning them at different points along a coordinate axis.
This increased expressiveness leads to significant feature efficiency, allowing NLTF to perform well even with a very small number of latent features ($R$). Furthermore, the nonlinear approach appears more robust on sparse datasets, where linear boundaries may be poorly defined due to lack of data points.
Experimental Evaluation and Results
The authors evaluated NLTF against state-of-the-art baselines, including PITF and SVD, across four datasets: Delicious-small, Last.fm, Movielens, and Delicious-large. The Delicious-large dataset, containing approximately 0.6 billion triples and 1.5 million users, served as a critical benchmark for scalability in "Big Data" scenarios where traditional Tucker-based models would fail.
NLTF consistently outperformed PITF and other baselines on the Delicious and Last.fm datasets. Notably, on the massive Delicious-large dataset, NLTF showed superior performance across all ranking metrics.
In low-rank scenarios (e.g., $R=4$), NLTF's performance advantage was even more pronounced, confirming that the Gaussian kernel utilizes the latent space more effectively than linear models. Qualitative analysis further validated the "multi-class" theory; when visualized in low dimensions, the model automatically clustered tags into intuitive groups such as "Oscar and AFI," "Movie Content," and "Adjectives."
Conclusion and Future Work
While NLTF showed exceptional performance on sparse and large-scale datasets, PITF maintained a slight edge on the denser Movielens dataset, suggesting that linear interactions may be sufficient for certain data distributions. Future research could explore incorporating content-based features (such as user profiles or item descriptions) to address the cold-start problem for new users and items. Additionally, automating the tuning of the kernel width parameter ($\beta$), which currently requires grid search, remains an area for optimization.
Summary updated