Augmenting Business Entities with Salient Terms from Twitter

Riham Mansour, Nesma Refaei, Vanessa MurdockCOLING 2014

TL;DR

This paper tackles the challenge of improving local search results by enriching business entity descriptions with relevant terms extracted from social media (Twitter) and the web. By augmenting business metadata with these "salient terms," the authors demonstrate a significant improvement in search result precision, particularly for queries that require a deeper understanding of a business's offerings beyond its basic category.

What Problem Does This Paper Address?

Local search engines, especially those accessed via mobile devices, often struggle to match user queries with the correct business listings. This is because the information indexed about businesses (name, category, address) may not align with the descriptive language users employ in their search queries. For instance, a user looking for "roasted chiles in Santa Fe" might not find relevant results if the business listing only categorizes itself as a "restaurant" and doesn't explicitly mention "roasted chiles." This mismatch is particularly problematic for "descriptive queries" that don't explicitly name a business or category. The paper aims to bridge this gap by creating richer, more descriptive representations of businesses.

Methods

The core of the paper's contribution lies in its method for extracting "salient terms" that characterize a business. These terms are derived from two primary sources:

  • Geo-tagged Tweets: Tweets posted within a one-kilometer radius of a business are considered. To identify salient terms, the authors compute term co-occurrence with the business name, but crucially, they use user frequency rather than raw term frequency. This prevents a few highly active users from disproportionately influencing the term distribution.
  • Web Pages: The top three search results from the Bing Search API for a business's name are analyzed. Salient terms are extracted from these pages, and their importance is weighted by term frequency (tf). Terms with tf > 0.001 are considered salient.

These extracted terms are then used to augment the existing business entity representation. The paper evaluates different combinations of these augmented representations (entity metadata alone, with Twitter terms, with web terms, and with both) in a local search retrieval task.

Datasets

  • Twitter Firehose: Geo-tagged tweets from the US and Brazil were collected from January 1, 2013, to May 31, 2013.
  • Proprietary Business Listings Database: A database of 30,000 businesses from the US and Brazil was used to create the search corpus.
  • Test Set: A subset of 80 businesses (40 from the US, 40 from Brazil) was selected to formulate test queries.
  • Query Generation: Query templates were created based on business attributes (name, location, category) and salient terms identified by judges. A total of 1000 local queries were generated and filtered using the Bing Search API to ensure they had a local intent.

Key Insights and Ideas

  • Social Media as a Rich Source: Geo-tagged social media, despite its inherent noise, provides a valuable source of descriptive terms that users actually employ when talking about businesses.
  • User Frequency Matters: Using user frequency for tweet analysis is crucial to avoid bias from prolific users and to capture a more representative distribution of how people describe places.
  • Web Pages are More Reliable: Web pages associated with businesses tend to provide more reliable and accurate salient terms compared to social media, likely due to their more curated nature.
  • Synergy of Sources: Combining terms from both social media and the web yields the best results, suggesting that these sources capture different, complementary aspects of a business's description.
  • Small Number of Tweets Suffice: Surprisingly, only 10-30 tweets were found to be sufficient for improving retrieval results, indicating that adding more tweets beyond a certain point might introduce more noise than signal.

Key Results

The paper demonstrates significant improvements in local search precision by augmenting business entities with salient terms:

  • U.S. Results:
    • Precision at rank one (P@1) improved by 18% when augmenting with both web terms and Twitter terms, compared to a baseline using only entity metadata.
  • Precision at rank three (P@3) improved by 19%.
  • Brazil Results:
    • P@1 improved by 9% with the combined augmentation.
  • P@3 improved by 15%.

The authors also conducted an independent evaluation of the quality of extracted terms, finding that web-derived terms had a higher accuracy (0.78 for US, 0.85 for Brazil) than Twitter-derived terms (0.22 for US, 0.15 for Brazil). However, the combined approach still showed the greatest benefit.

Limitations and Future Work

  • Limitations:
    • The study focused on a limited set of 80 businesses and 1000 queries.
  • The evaluation of term quality showed a significant difference in accuracy between web and Twitter sources.
  • The paper acknowledges that the baseline system (Colombo et al., 2013) used term frequency, whereas their proposed method uses user frequency for tweets, which is a methodological difference.
  • Future Work:
    • Applying query expansion from social media to larger collections of local search queries.
  • Exploring other methods for formulating query templates based on available business metadata.
  • Investigating further methods for improving the extraction and representation of salient terms.

Summary updated

BibTeX
@inproceedings{mansour2014augmenting,
  author = {Riham Mansour and Nesma Refaei and Vanessa Murdock},
  title = {{Augmenting Business Entities with Salient Terms from Twitter}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2014},
}
Figure