Plugging Small Models in Large Language Models for POI Recommendation in Smart Tourism

Hong Zheng, Zhenhui Xu, Qihong Pan, Zhenzhen Zhao, Xiangjie Kong

TL;DR

This paper introduces PSLM4ST, a novel framework that enhances Large Language Models (LLMs) for Point-of-Interest (POI) recommendation in smart tourism. By integrating a lightweight temporal knowledge graph (TKG) plugin, PSLM4ST effectively captures fine-grained user preferences and temporal dynamics, leading to more accurate and personalized recommendations.

What Problem Does This Paper Address?

Recommending Points of Interest (POIs) is crucial for smart tourism, aiming to provide personalized experiences for travelers. While Large Language Models (LLMs) have shown promise due to their common-sense reasoning abilities, existing LLM-based approaches often struggle with:

  • Limited Context Window: LLMs can't process entire user trajectories at once, potentially missing crucial long-term patterns.
  • Insufficient Preference Capture: They may not fully grasp nuanced user preferences, especially when relying solely on limited input.
  • Ignoring Temporal Dynamics: Crucial temporal information, like specific time slots for activities, is often inadequately addressed, hindering effective travel planning.
  • Lack of Interpretability: Traditional deep learning models often lack clear explanations for their recommendations.

This paper aims to bridge these gaps by creating a system that leverages the strengths of both LLMs and specialized models to overcome these limitations.

Methods

The core innovation of PSLM4ST lies in its "plugin" architecture, where a lightweight temporal knowledge graph (TKG) reasoning model acts as a plugin for LLMs. This approach involves several key components:

  • Temporal Knowledge Graph (TKG): A dynamic graph that captures check-in behavior over time, allowing for the modeling of temporal patterns and relationships between POIs.
  • Plugin Model: This module is designed to extract fine-grained user preferences and temporal information. It comprises:
  • User Personal Habit and Novelty Preferences: Distinguishes between users' tendency to visit familiar places versus exploring new ones.
  • Personal and Global POI Transfer Preferences: Models how users move between POIs, considering both individual habits and broader collaborative patterns.
  • Mirror Modules: Captures preferences related to POI categories and geographical regions.
  • User Profile Generation: LLMs are used to generate a comprehensive user profile, including basic attributes (gender, age, education, income) and a detailed summary of user behavior patterns.
  • Plugin-Enhanced Prompt: The preliminary predictions from the plugin model, along with the user profile and trajectory, are used to construct a detailed prompt for the LLM. This prompt guides the LLM to perform more accurate reasoning.
  • LLM Fine-Tuning: Parameter-efficient fine-tuning (PEFT) is employed to adapt a base LLM (Llama-2-7b-longlora-32k) for the recommendation task.

The synergy between the plugin model (providing fine-grained, temporally aware candidate sets) and the LLM (offering common-sense reasoning and contextual understanding) is central to PSLM4ST's effectiveness.

Datasets

The model was evaluated on three widely adopted real-world datasets:

  • Foursquare-NYC: Check-in data from New York City.
  • Foursquare-TKY: Check-in data from Tokyo.
  • Gowalla-CA: Check-in data from California.

These datasets were preprocessed by sorting records chronologically, splitting them into training (80%), validation (10%), and test (10%) sets, and filtering out users/POIs with fewer than 10 check-ins.

Key Insights and Ideas

  • Synergy is Key: The paper strongly advocates for the combined power of specialized "plugin" models and general-purpose LLMs. The plugin model provides the LLM with structured, contextually relevant information, while the LLM offers broad reasoning capabilities.
  • Temporal Dynamics Matter: The analysis clearly shows that user check-in behavior is highly dependent on time slots. Ignoring this leads to suboptimal recommendations, and PSLM4ST's TKG-based approach effectively addresses this.
  • Fine-Grained Preferences are Crucial: Beyond general interests, capturing subtle preferences like habit vs. novelty, and transfer patterns between POIs, significantly improves recommendation accuracy.
  • User Profile Enrichment: Using LLMs to generate detailed user profiles, including summaries of behavior and attributes, provides richer context for recommendation.

Key Results

PSLM4ST demonstrated superior performance across the evaluated datasets compared to both traditional deep learning methods and other LLM-based baselines.

  • Overall Performance: PSLM4ST consistently outperformed baselines in metrics like Acc@1, Acc@5, Acc@10, and MRR on the NYC, TKY, and CA datasets. For example, on the NYC dataset, PSLM4ST achieved an Acc@1 of 0.3388, significantly higher than most baselines.
  • LLM-Based Comparison: Compared to LLM-based methods like LLM-ZS and GenUP, PSLM4ST achieved higher Acc@1 scores (e.g., 0.3388 vs. 0.2575 for GenUP on NYC).
  • Plugin Model Effectiveness: Ablation studies showed that removing the plugin model significantly degraded performance, highlighting its critical role in providing accurate candidate sets. The user profile summary was found to have a greater impact than basic user attributes.
  • Module Contributions: The analysis of preliminary predictions showed that while personal preferences (Habit, Novelty, Transfer) are important, global POI transfer (GTF) was particularly effective on datasets with less individual user data (like CA), suggesting its robustness.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Empirical Hyperparameters: The model relies on empirical hyperparameters (like α and β) to balance different preference modules. This requires manual tuning for different datasets, increasing complexity. Future work aims to explore adaptive mechanisms for automatic tuning.
  • Multimodal Data Integration: The current work focuses on check-in data. Future research plans to integrate more multimodal data sources, such as text reviews, images, videos, social relationships, and POI descriptions, to enrich POI information and user understanding.
  • Computational Cost: The architecture involves both LLM and plugin model inference, which can be computationally intensive. The paper notes the need for plugin models to have higher inference efficiency.

Summary updated

Abstract

Point-of-interest (POI) recommendation is a crucial task in location-based social networks, especially for enhancing personalized travel experiences in smart tourism. Recently, large language models (LLMs) have demonstrated significant potential in this domain. Unlike classical deep learning-based methods, which focus on capturing various user preferences, LLM-based approaches can further analyze candidate POIs using common sense and provide corresponding reasons. However, existing methods often fail to fully capture user preferences due to limited contextual inputs and insufficient incorporation of cooperative signals. Additionally, most methods inadequately address target temporal information, which is essential for planning travel itineraries. To address these limitations, we propose PSLM4ST, a novel framework that enables synergistic interaction between LLMs and a lightweight temporal knowledge graph reasoning model. This plugin model enhances the input to LLMs by making adjustments and additions, guiding them to focus on reasoning processes related to fine-grained preferences and temporal information. Extensive experiments on three real-world datasets demonstrate the efficacy of PSLM4ST.
BibTeX
@article{zheng2025plugging,
  author = {Hong Zheng and Zhenhui Xu and Qihong Pan and Zhenzhen Zhao and Xiangjie Kong},
  title = {{Plugging Small Models in Large Language Models for POI Recommendation in Smart Tourism}},
  journal = {Algorithms},
  year = {2025},
  doi = {10.3390/a18070376},
}
Figure