TL;DR
This paper introduces TF-DCon, a novel training-free method for dataset condensation specifically for content-based recommendation (CBR) systems. By leveraging ChatGPT for item content summarization and a clustering-based approach for user synthesis, TF-DCon significantly reduces dataset size (up to 95%) while maintaining high recommendation performance (up to 97% of original) and drastically improving training efficiency.
What Problem Does This Paper Address?
Modern content-based recommendation (CBR) systems excel at personalizing recommendations by utilizing rich item content. However, training these models on massive datasets is computationally expensive and time-consuming, especially with the need for frequent retraining. Existing dataset condensation techniques, which aim to create smaller, representative datasets, struggle with the discrete nature of textual data and the preservation of crucial user preference information inherent in recommendation tasks. This paper tackles the challenge of effectively condensing textual CBR datasets without requiring extensive retraining of the condensation model itself.
Methods
The proposed TF-DCon method operates in two main stages:
- Content-Level Condensation: This stage uses ChatGPT to condense the rich textual content of each item (title, abstract, category) into a concise, informative title. To ensure optimal summarization, a "prompt evolution" module is employed. This module iteratively refines prompts given to ChatGPT, guiding it to generate titles that best preserve the original item's information, measured by embedding similarity.
- User-Level Condensation: This stage addresses the challenge of preserving user preferences. It first uses ChatGPT to extract user interests from their historical interactions. Then, a clustering-based synthesis module is used. User embeddings are clustered, and each cluster represents a synthetic user. Interactions for these synthetic users are generated by merging the historical interactions of real users within each cluster, prioritizing those with similar interests and embeddings.
A key innovation is that TF-DCon is "training-free," meaning it doesn't require iterative training of the condensation model on the condensed data, unlike many prior methods. This significantly boosts efficiency.
Datasets
The authors evaluated TF-DCon on three real-world datasets:
- MIND: A large-scale dataset for news recommendation.
- Goodreads: A dataset for book recommendation.
- MovieLens: A dataset for movie recommendation.
These datasets were split into 80% for training, 10% for validation, and 10% for testing. The condensation was performed on the training sets, while testing used the original test sets.
Key Insights and Ideas
- ChatGPT as a Content Condenser: The paper demonstrates that LLMs like ChatGPT can be effectively leveraged to summarize textual item content into concise titles, preserving essential information for recommendation models.
- Prompt Evolution for Optimization: The prompt evolution mechanism is crucial for adapting LLMs to the specific domain of recommendation and ensuring high-quality content condensation.
- Clustering for User Preference Preservation: The user-level condensation, using clustering and interest extraction, is vital for capturing and synthesizing user preferences, a critical aspect often overlooked by general dataset condensation methods.
- Training-Free Efficiency: The "training-free" nature of TF-DCon is a significant advantage, drastically reducing the computational overhead associated with dataset condensation.
Key Results
- Performance: TF-DCon achieved performance comparable to models trained on full datasets, approximating up to 97% of the original performance on the MIND dataset.
- Dataset Size Reduction: The method achieved substantial dataset size reduction, up to 95% (e.g., 98% reduction on the user side for MIND).
- Training Efficiency: TF-DCon enabled training speeds up to 5x faster compared to training on original datasets, with convergence in minutes rather than hours.
- Generalization: Condensed datasets generated by TF-DCon showed good generalization capabilities across different CBR model architectures (NAML, NRMS, Fastformer).
- LLM Versatility: While ChatGPT performed best, the method showed promising results even with smaller LLMs like Llama-2-7b.
Limitations and Future Work
- LLM Dependency and Latency: The reliance on ChatGPT for pre-processing introduces potential instability due to network latency and a linear cost scaling with dataset size.
- Cost-Gain Balance: Achieving a cost-gain balance might require training on condensed datasets for a certain duration.
- Open-Source LLM Performance: The performance of TF-DCon with open-source LLMs was noted as being limited, suggesting room for improvement.
The authors suggest future work could explore more efficient LLM integration and further investigate the balance between condensation ratio and performance.
Summary updated