Understanding the sentiment associated with cultural ecosystem services using images and text from social media

Ilan Havinga, Diego Marcos, P.w. Bogaart, Devis Tuia, Lars HeinEcosystem 2023 · DOI

TL;DR

This paper leverages social media data (Flickr images and text) to develop more nuanced indicators of Cultural Ecosystem Services (CES) by combining computer vision and natural language processing. By analyzing both visual content and user sentiment, the study provides a richer understanding of how people experience nature, finding that aesthetic quality and wildlife presence correlate with positive sentiment, but social activities also play a significant role.

What Problem Does This Paper Address?

Cultural Ecosystem Services (CES) represent the non-material benefits humans derive from nature, such as aesthetic appreciation and recreation. While social media platforms like Flickr offer a vast source of data for studying CES, previous large-scale analyses have often relied on simple proxies like the number of images. This approach overlooks the crucial aspect of user experience and the sentiment associated with these interactions. The challenge lies in processing the sheer volume of data to extract meaningful insights beyond mere presence, and in validating these novel data sources against traditional methods. This paper aims to bridge this gap by integrating image and text analysis to capture the sentiment and experience associated with CES.

Methods

The study employs a multi-modal approach combining Computer Vision (CV) and Natural Language Processing (NLP) techniques.

  • CV for CES:
    • Aesthetic Quality: A Convolutional Neural Network (CNN) trained on the ScenicOrNot dataset was used to predict aesthetic quality ratings for landscape images.
  • Wildlife Observations: Another CNN, trained on Flickr and iNaturalist images, was used to identify wildlife observations.
  • NLP for Sentiment Analysis: Three NLP models were applied to the text associated with Flickr images (titles, tags, descriptions):
  • Hedonometer: A bag-of-words approach using a crowdsourced sentiment lexicon.
  • Sentiment140: A Maximum Entropy classifier trained on tweets.
  • RoBERTa: A deep learning model fine-tuned for sentiment analysis on tweets.
  • Adjective Analysis: A tf-idf (term frequency-inverse document frequency) analysis was performed to identify uniquely used adjectives associated with specific species and aesthetic ratings.
  • Physical Settings and Activities: The Places365 model was used to classify image scene classes and attributes, which were then analyzed in conjunction with sentiment.
  • Validation: The Flickr-based CES and sentiment measures were compared against a national survey of recreational well-being in England (MENE survey) to validate the findings.

Datasets

  • Flickr Image Dataset: A collection of 9.8 million outdoor images from Great Britain.
  • ScenicOrNot Dataset: Used for training the aesthetic quality prediction model.
  • iNaturalist Dataset: Used in conjunction with Flickr images for training the wildlife observation model.
  • Monitor of Engagement with the Natural Environment (MENE) Survey: A national survey of nature recreation in England, used for validation.

Key Insights and Ideas

  • Sentiment Matters: Simply counting images of natural landscapes or wildlife is insufficient; the sentiment expressed by users provides a crucial layer of understanding about the value of these experiences.
  • Aesthetics Drive Positivity: Higher aesthetic quality in landscapes consistently correlates with more positive sentiment, reinforcing the importance of visually appealing natural environments for well-being.
  • Wildlife's Emotional Impact: The presence of specific wildlife is linked to positive sentiment, but the type of wildlife matters, with plants and birds generally eliciting more positive reactions than insects or arachnids.
  • Social Context is Key: Sentiment is not solely driven by natural elements; social activities like "socializing" and "campsites" are strongly associated with positive sentiment, highlighting the role of human interaction in nature.
  • Bridging the Gap: The study demonstrates that social media data, when analyzed with advanced CV and NLP techniques, can provide a valuable and scalable complement to traditional survey methods for understanding CES.

Key Results

  • All three NLP models showed a significant positive trend between aesthetic quality and sentiment.
  • Positive adjectives like "beautiful" and "calm" were more prevalent in images with higher aesthetic ratings, while negative adjectives like "ugly" and "sad" decreased.
  • Plants, particularly flowering species, generated the most positive sentiment, followed by birds and insects. Spiders and scorpions received the least positive sentiment.
  • Attributes like "flowers" and "socializing" were linked to the most positive sentiment, while "railroad" and "brick" were associated with the least.
  • The study found significant correlations between Flickr-based CES measures (aesthetic quality, wildlife observations) and sentiment, and the MENE survey responses, particularly for statements like "I enjoyed it" and "I felt close to nature." For instance, trips taken to enjoy scenery had a higher mean aesthetic quality (4.29) in Flickr images compared to trips not taken for scenery (3.19).

Limitations and Future Work

  • Data Bias: Social media data may not be representative of all demographics or experiences, potentially underrepresenting contemplative or less overtly positive emotions.
  • Model Specificity: The performance of NLP models can be influenced by their training data (e.g., Twitter vs. Flickr), and some models may struggle with nuanced language.
  • Contextual Understanding: Certain experiences, like the restorative effects of cemeteries, are difficult to capture with current sentiment analysis tools and require more contextual information.
  • Future Work: The authors suggest further research into developing integrated CES indicators by combining sentiment and aesthetic quality scores, and exploring the use of more diverse training datasets to capture a wider range of human-nature interactions. They also highlight the need for continued validation of social media-based indicators against survey data.

Summary updated

Abstract

Social media is increasingly being employed to develop Cultural Ecosystem Services (CES) indicators. The image-sharing platform Flickr has been one of the most popular sources of data. Most large-scale studies, however, tend to only use the number of images as a proxy for CES due to the challenges associated with processing large amounts of this data but this does not fully represent the benefit generated by ecosystems in terms of the positive experiences expressed by users in the associated text. To address this gap, we apply several Computer Vision (CV) and natural language processing (NLP) models to link CES estimates for Great Britain based on the content of images to sentiment measures using the accompanying text, and compare our results to a national, geo-referenced survey of recreational well-being in England. We find that the aesthetic quality of the landscape and the presence of particular wildlife results in more positive sentiment. However, we also find that different physical settings correlate with this sentiment and that sentiment is sometimes more strongly related to social activities than many natural factors. Still, we find significant associations between these CES measures, sentiment and survey data. Our findings illustrate that integrating sentiment analysis with CES measurement can capture some of the positive benefits associated with CES using social media. The additional detail provided by these novel techniques can help to develop more meaningful CES indicators for recreational land use management.
BibTeX
@inproceedings{havinga2023understanding,
  author = {Ilan Havinga and Diego Marcos and P.W. Bogaart and Devis Tuia and Lars Hein},
  title = {{Understanding the sentiment associated with cultural ecosystem services using images and text from social media}},
  booktitle = {Ecosystem},
  year = {2023},
  doi = {10.1016/j.ecoser.2023.101581},
}
Figure