Personalized Landmark Recommendation Based on Geotags from Photo Sharing Sites

Yue Shi, Pavel Serdyukov, Alan Hanjalić, Martha Larson

TL;DR

Researchers have developed a "Category-Regularized Matrix Factorization" (CRMF) approach that uses geotagged Flickr photos and Wikipedia categories to recommend personalized landmarks. Unlike standard systems that just suggest the most popular sites, this model excels at finding "hidden gems" tailored to a traveler's specific tastes by leveraging the semantic categories of places they’ve visited before.

What Problem Does This Paper Address?

When you visit a new city like Paris, you don't need an algorithm to tell you to visit the Eiffel Tower—you already know it exists. The real value in a travel recommendation system lies in finding landmarks that are less famous but perfectly aligned with your personal interests (e.g., a specific style of Gothic architecture or obscure contemporary art galleries).

However, building such a system is difficult due to two main challenges:

  1. Extreme Data Sparsity: Most travelers only visit a tiny fraction of the world's landmarks. In the authors' dataset, the user-landmark matrix was 99.89% empty.
  2. The Popularity Bias: Standard recommendation algorithms are often dominated by "hits." In travel, this means they default to recommending the most-visited "tourist traps," which provides little utility to the user.

Methods

The authors frame landmark recommendation as a Collaborative Filtering (CF) problem but introduce a novel twist called Category-Regularized Matrix Factorization (CRMF).

  • The Standard Approach: Basic Matrix Factorization (MF) decomposes a user-item matrix into latent features to predict missing entries. If User A and User B both liked Landmark X, the system assumes they might both like Landmark Y.
  • The Innovation: CRMF adds a "regularization" term based on landmark categories sourced from Wikipedia. Even if two landmarks have never been visited by the same person, the system can recognize they are similar if they share categories (e.g., "14th-century churches").
  • The Logic: By integrating category-based similarity directly into the learning process, the model can "bridge" the gaps in sparse data. It learns a user’s preference for specific types of places across different cities, rather than just looking for overlapping user histories.

Datasets

The study utilized two primary data sources:

  • Flickr: The researchers crawled metadata for 42.9 million geotagged photos. After filtering for "travelers" (users visiting at least two cities outside the US), they ended up with a dataset of 126,123 photos from 40,084 users across 40 major cities.
  • Wikipedia: They mapped geotags to Wikipedia articles to identify landmarks and extracted 7,379 distinct categories to build a landmark-category matrix.

Key Insights and Ideas

  • The "Eiffel Tower" Problem: The authors argue that a recommendation's utility is inversely proportional to its popularity. A good system should be evaluated on its ability to find the "long tail" of landmarks.
  • Power-Law Reality: Both users and landmarks follow a power-law distribution. A few landmarks get almost all the "foot traffic," while the vast majority are rarely visited. This confirms why simple popularity-based models (PopRec) often seem successful on paper but fail to provide personalized value.
  • Semantic Bridging: Using Wikipedia categories allows the model to perform "cross-city" recommendations more effectively. If the system knows you like "Art Nouveau" in Prague, it can find "Art Nouveau" in Barcelona, even if the user communities for those two specific landmarks don't overlap much in the training data.

Key Results

The researchers evaluated their model using Mean Average Precision (MAP) and compared it against a popularity-based baseline (PopRec) and standard Matrix Factorization (MF).

  • When including popular landmarks: PopRec actually performed best. This is because, statistically, most people do visit the most popular sites.
  • When ignoring "Tourist Traps": To test true personalization, the authors removed the top $N$ most popular landmarks from each city. As $N$ increased (i.e., as the task got harder and more focused on the "long tail"), CRMF’s superiority became clear.
  • Performance Gains: When the top 10 most popular landmarks were ignored, CRMF outperformed PopRec by 31.6% and standard MF by 11.7%. This demonstrates that category regularization is essential for overcoming data sparsity in personalized niches.

Limitations and Future Work

  • Cold Start: While the model handles sparse data well, it still requires some initial geotagged history from a user to make a recommendation.
  • Contextual Blindness: The current model doesn't account for the order of visits or the time spent at a location, which could indicate the strength of a user's preference.
  • Future Directions: The authors suggest integrating more diverse information, such as social network connections between users and the specific "travel experience" level of a user (e.g., distinguishing between a first-time backpacker and a seasoned traveler).

Summary updated

Abstract

Geotagged photos of users on social media sites provide abundant location-based data, which can be exploited for various location-based services, such as travel recommendation. In this paper, we propose a novel approach to a new application, i.e., personalized landmark recommendation based on users’ geotagged photos. We formulate the landmark recommendation task as a collaborative filtering problem, for which we propose a category-regularized matrix factorization approach that integrates both user-landmark preference and category-based landmark similarity. We collected geotagged photos from Flickr and landmark categories from Wikipedia for our experiments. Our experimental results demonstrate that the proposed approach outperforms popularity-based landmark recommendation and a basic matrix factorization approach in recommending personalized landmarks that are less visited by the population as a whole.
BibTeX
@inproceedings{shi2021personalized,
  author = {Yue Shi and Pavel Serdyukov and Alan Hanjalić and Martha Larson},
  title = {{Personalized Landmark Recommendation Based on Geotags from Photo Sharing Sites}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/icwsm.v5i1.14152},
}
Figure