Unsupervised Sustainability Report Labeling based on the integration of the GRI and SDG standards

Seyed Alireza Mousavian Anaraki, Danilo Croce, Roberto BasiliACL 2025

TL;DR

This paper introduces an unsupervised method to automatically label paragraphs in corporate sustainability reports with relevant Global Reporting Initiative (GRI) and Sustainable Development Goals (SDG) standards. By combining report structure, official mappings, and semantic similarity, it creates a weakly supervised dataset that improves downstream classification tasks, offering a scalable solution for analyzing corporate sustainability claims.

What Problem Does This Paper Address?

Corporations are increasingly expected to report on their sustainability efforts, often using frameworks like the Global Reporting Initiative (GRI) and the United Nations' Sustainable Development Goals (SDGs). However, these reports are lengthy, complex, and unstructured, making manual analysis for alignment with these frameworks incredibly time-consuming, expensive, and prone to errors. Existing automated methods often rely on extensive manual annotation, which limits scalability. This paper aims to bridge this gap by developing an unsupervised approach to automatically identify and label relevant paragraphs within these reports, reducing the burden on human experts and enabling large-scale analysis.

Methods

The core of the proposed method is an unsupervised pipeline that leverages multiple sources of information:

  • GRI Content Indexes: The paper utilizes the structured GRI content indexes found within sustainability reports. These indexes map specific GRI codes to page ranges where they are discussed, providing an initial, albeit potentially incomplete, set of candidate labels for paragraphs.
  • GRI-SDG Mappings: Official mappings between GRI codes and SDG targets are used to expand the potential labels. This step helps to connect specific corporate disclosures to broader global sustainability goals.
  • Semantic Similarity: Pre-trained sentence embedding models (specifically, all-mpnet-base-v2) are employed to measure the semantic similarity between paragraphs and the textual descriptions of GRI disclosures and SDG targets. This allows for the ranking of potential label assignments.
  • Disambiguation: A key innovation is the disambiguation step, which uses a scoring mechanism (Wpgs = σ(p, g) · σ(p, s)) to select the most plausible GRI-SDG label pair for each paragraph. This process considers both candidate labels derived from the GRI index and alternative labels identified through broader semantic similarity, aiming to balance precision and recall.

Datasets

  • Training Data: The authors created a weakly supervised dataset by applying their unsupervised pipeline to 30 sustainability reports from 2023, spanning 10 industry sectors. This resulted in 19,133 paragraphs, with 10,303 paragraphs falling within GRI index page ranges and thus being candidates for annotation.
  • Evaluation Data: The Open Source SDG (OSDG) Community Dataset was used for evaluation. This dataset contains paragraphs labeled with a single SDG and validated by multiple annotators, providing a benchmark for assessing the quality of the automatically generated labels. The authors created "simple" (full agreement) and "complex" (partial consensus) test sets from OSDG.

Key Insights and Ideas

  • Leveraging Report Structure is Crucial: The GRI content index, despite its potential biases, provides a valuable starting point for unsupervised annotation, significantly narrowing the search space.
  • Bridging Frameworks is Key: The explicit mapping between GRI and SDGs is essential for connecting corporate-specific disclosures to global sustainability agendas, making the annotations more meaningful.
  • Semantic Similarity is Powerful but Needs Context: While sentence embeddings capture semantic alignment, the disambiguation step is critical to avoid noise and select the most relevant labels, especially when alternative labels show high similarity.
  • Unsupervised Data Can Enhance Supervised Models: The study demonstrates that automatically generated, weakly supervised data can improve the performance of downstream classification models, even when trained on a different domain (corporate reports vs. general policy texts). This suggests that unsupervised methods can effectively augment existing labeled datasets.

Key Results

  • When training a BERT-based multi-label classifier solely on the auto-labeled data, the model achieved an accuracy of 0.762 on the simple OSDG test set and 0.737 on the complex test set.
  • Combining the auto-labeled data with the full OSDG dataset yielded the best performance, achieving 0.924 accuracy on the simple test set and 0.910 on the complex test set. This indicates that the auto-labeled data contributes positively without introducing significant noise.
  • An inverse evaluation (training on OSDG and testing on auto-labeled data) resulted in a significantly lower accuracy of 40%, highlighting the domain shift between the datasets and the robustness of the proposed method when trained on company-specific data.

Limitations and Future Work

  • Evaluation Limitations: The current evaluation relies on indirect metrics and a single-label setting for OSDG, which doesn't fully capture the multi-label nature of sustainability disclosures. GRI annotations were not evaluated due to the lack of ground-truth data.
  • Quality vs. Topic: The method identifies relevant topics but does not assess the quality or truthfulness of the claims, which is crucial for distinguishing genuine reporting from "greenwashing."
  • Paragraph as Unit: Overlapping disclosures within paragraphs are not explicitly handled, treating each paragraph as an independent unit.
  • Reliance on GRI Indexes: The approach depends on company-provided GRI indexes, which may reflect selective transparency or bias.
  • Future Work: The authors propose several directions:
  • Directly evaluating GRI annotations.
  • Developing more nuanced evaluation strategies, including multi-label classification and uncertainty-aware models.
  • Assessing the nature of reported content to detect subtle forms of greenwashing and evaluate the substance and credibility of claims.
  • Extending the scope to include broader ESG reporting.

Summary updated

BibTeX
@inproceedings{anaraki2025unsupervised,
  author = {Seyed Alireza Mousavian Anaraki and Danilo Croce and Roberto Basili},
  title = {{Unsupervised Sustainability Report Labeling based on the integration of the GRI and SDG standards}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
}

Loading related content...

Figure