Feeling travel photographs: Emotional labelling in machine-generated semantic differentials

Joanne Yu, Denis Helić, Astrid Dickinger, Markus StrohmaierResearch Square 2025 · DOI

TL;DR

Researchers have developed a machine-learning pipeline to "read" the emotional vibes of over 33,000 Instagram travel photos, mapping them onto a scale of pleasure (hedonic) versus practicality (utilitarian). The study reveals that "fun" isn't a universal engagement driver; while thrill-seeking works for cycling adventures, it actually turns users off when applied to dining or serene riverside scenery.

What Problem Does This Paper Address?

In the world of digital tourism marketing, the prevailing wisdom is that "hedonic" content—images that evoke pleasure, fun, and excitement—is the gold standard for engagement. However, measuring these intangible emotional qualities objectively has always been a hurdle.

Traditionally, researchers used "semantic differentials," which are surveys asking people to rate things on a scale of polar opposites (e.g., "Exciting" vs. "Dull"). This method is slow, expensive, and prone to "social desirability bias," where people answer how they think they should rather than how they feel. This paper seeks to automate this process using AI to understand how different emotional dimensions—both fun and functional—impact likes and comments across various travel contexts.

Methods

The authors employed a sophisticated multi-step data science pipeline:

  1. Image Captioning: Instead of using simple object labels (like "bus" or "tree"), they used Open Clip Torch to generate descriptive, narrative captions (e.g., "a double-decker bus driving past a clock tower").
  2. Topic Modeling: They used BERTopics (which combines Sentence Transformers, UMAP for dimensionality reduction, and K-Means clustering) to categorize the images into 15 distinct themes, such as "Dining Ambience," "Castle Architecture," and "Cycling Adventures."
  3. Machine-Generated Semantic Differentials: This is the core innovation. They took 24 bipolar emotional scales (like fun–not fun or functional–not functional) and used WordNet definitions to create vector representations of these "poles." By projecting the image captions into this 24-dimensional vector space, they could mathematically "score" how much an image leaned toward one emotion or another.
  4. Engagement Analysis: Finally, they used Negative Binomial Regression to see how these emotional scores correlated with likes and comments, while controlling for seasonality and follower counts.

Datasets

The study analyzed a dataset of 33,001 Instagram posts from the official destination marketing accounts of 10 major European cities (including London, Paris, Berlin, and Vienna). The data spanned from 2016 to 2022 to ensure a robust sample size and account for seasonality.

Key Insights and Ideas

  • The "Fun" Fallacy: One of the most surprising "aha moments" was that "fun" consumption actually correlated negatively with likes for "multimodal excursions" (transportation like trains and boats). In these contexts, users seem to prefer informative, practical content over forced entertainment.
  • Contextual Thrills: "Thrill" is a double-edged sword. It significantly boosts engagement for cycling adventures, but it has a negative impact on "Riverside Charm" and "Dining Ambience." If you're posting a picture of a peaceful dinner, adding a "thrilling" vibe actually discourages people from commenting.
  • The Utility of Art: We usually think of art and sculpture as purely hedonic (for pleasure). However, the study found that images of art that also conveyed a sense of "functionality" or "usefulness" (perhaps by helping the viewer understand the piece) saw a boost in likes.
  • Productive Tranquility: Users showed high appreciation for "Cityscape and Tranquil Scenery" when it connoted "productivity." The authors suggest that engaging with serene content might foster a positive mental state that feels like a personal accomplishment or "mental productivity" for the user.

Key Results

  • Seasonality: Winter posts (October–March) generally received higher engagement rates than summer posts, likely due to the high emotional resonance of holidays like Christmas and New Year in Europe.
  • Functional vs. Hedonic: While "thrill" generally increased likes ($p < .001$), "functional" positioning often had a significant negative impact on the raw number of likes and comments across the board, unless specifically tailored to the right topic (like Art).
  • Topic Influence: Most topics influenced likes positively compared to the baseline ("Outdoor Gatherings"), with "Railway Exploration" and "Elevated Urban Transit" being particularly strong drivers of engagement.

Limitations and Future Work

The authors acknowledge several areas for improvement:

  • Geographic Bias: The study focused exclusively on European cities. Cultural perceptions of "fun" or "utility" might differ significantly in other regions.
  • The Marketer’s Lens: The data comes from official marketing accounts. The "vibe" of a tourist's personal photo might trigger very different engagement patterns.
  • Static vs. Dynamic: The study only looked at still photographs. With the massive rise of Reels and short-form video, future research needs to address how movement and sound alter these emotional differentials.
  • Sentiment Depth: While the study tracked how many comments a post got, it didn't analyze what those comments said. Future work could use sentiment analysis to see if "thrilling" photos elicit more positive or negative types of engagement.

Summary updated

Abstract

<title>Abstract</title> The pervasive rise of social media has placed visual content, especially photographs, at the forefront of digital marketing in tourism. As marketers strategically leverage emotionally resonant content on platforms, this research explores the relationship between hedonic and utilitarian dimensions embedded in pictorial representations and their impact on user engagement in tourism. By utilising a machine-generated semantic differential, this study analyses 33,001 picture-based posts from popular destinations on Instagram. The findings of this study reveal that user engagement dynamics vary across themes, with multimodal excursions (e.g., modes of transportation) emphasising informativeness over fun and thrill, while art and sculpture benefit from the integration of functionality. By using an interdisciplinary approach, this research explores the emotional connotations of visual content on the engagement of potential tourists. Overall, this paper underscores the importance of tailoring content strategies to specific thematic contexts and challenges preconceived notions about the universal appeal of certain pictorial elements.
BibTeX
@inproceedings{yu2025feeling,
  author = {Joanne Yu and Denis Helić and Astrid Dickinger and Markus Strohmaier},
  title = {{Feeling travel photographs: Emotional labelling in machine-generated semantic differentials}},
  booktitle = {Research Square},
  year = {2025},
  doi = {10.21203/rs.3.rs-7591283/v1},
}
Figure